Comment by mosselman

I've been building my own metasearch engine after using Kagi for a long while and I must say that my results are getting better even than theirs.

Kagi can do things at their scale that are not feasible for me right now and one of them is categorising links like they probably do for this feature. So it is inspiring. A route that might work is checking if there are ublock lists that keep track of paywalled sites.

I've also been looking into creating my own search index, but this is a huge undertaking. I might go the tiny index route like Marginalia has done and try to find a niche for my specific use cases.

Another route could be to categorise websites better automatically and build up a category index.