← Back to context

Comment by zozbot234

3 hours ago

> Many people don’t notice, but even Wikipedia has no normalization between articles in different languages. The language button there acts like its showing you a translated version of the article but its actually a completely different Encyclopedia and community of editors with no cross reference to the other language’s article and references at all.

The Wikipedia folks are working on language-independent, machine-readable structured representation of encyclopedic text (probably relying on something very much like frame semantics, via some sort of general compositional structure) in order to address this issue - see Abstract Wikipedia but note that the project is still at a very early, highly experimental stage. LLMs are not considered adequate for this task because they are non-deterministic and not auditable by humans, hence why a different approach is being planned.

LLMs would be good enough until this project got anywhere

Don’t let perfect get in the way of good

I don’t trust Wikipedia to be incentivized to do this, and looking at the stub about it I have even less confidence

https://en.wikipedia.org/wiki/Abstract_Wikipedia

the last reference is from 2023 before this evolutionary branch of transformers at all