Comment by asciimoo
1 month ago
Wow, this is a really inspiring use case and blog post. Thanks for sharing it.
What tools or features would Hister need to support your complete search workflow?
1 month ago
Wow, this is a really inspiring use case and blog post. Thanks for sharing it.
What tools or features would Hister need to support your complete search workflow?
Thank you so much for building this! It's a really awesome piece of work and I'm grateful for your work. I'd love to sponsor you on Github in the near future.
The only thing that I think would be interesting to see is native support for crawling via a sitemap.xml instead of recursively. I worked around this by implementing a basic scraper that fetched pages exclusively from the sitemap.xml to add into Hister.
I think you're already aware of this, but I also experienced some data loss during the import because I was running a concurrent reindex. I clocked it pretty quickly so I didn't think too much of it. [1]
[1] "TODO store new documents in both indexes while running reindex to guarantee not losing any data." @ https://github.com/asciimoo/hister/blob/master/server/indexe...