Comment by progval
5 days ago
Zulip seems to rely too heavily on Javascript to be indexable by search engines. I copy-pasted some sentences from month-old posts on https://leanprover.zulipchat.com/ into Google and Bing, and neither could find the posts.
Search engines are great at rendering content using JavaScript, they're just not able to explore the Zulip web application's URL structure, so they tend to index one page per Zulip installation.
The current solution is that projects that want their publicly available content to be search engine indexed set up https://github.com/zulip/zulip-archive, which usually involves hooking a GitHub action up to GitHub Pages for a 0-infrastructure deployment. Ultimately, that's the same model as an IRC indexing project like this one: a separate tool from the chat server is responsible for search engine indexing.
Lean runs one here: https://leanprover-community.github.io/archive/, but it looks like it hasn't updated in a year, so likely whoever in the Lean community needs to investigate why it isn't updating.
https://github.com/zulip/zulip/issues/21881 tracks our goal of making the server natively offer search engine indexing. The current separate archive tool model has some advantages (search engine load can't break the server, for example), but I think it'll be worth doing the native version when we can find the resources for it.
Source: I lead the Zulip project.