Comment by adsharma
9 hours ago
So many words. But missing the one that matters the most: Explainability.
AI slowdown is worth it only if it can be made more explainable.
Changing the language we use to discuss it is a good first step.
We need to stop using inside baseball terms like alignment and mechanistic interpretability. Replace them with explainable tech. Graph Databases, Causality, Shared semantic spaces.
Previous writings on the topic (also on LinkedIn, but can't find urls):
https://x.com/arundsharma/status/2005338775468282339 https://x.com/latentpedia
As much as I love all these technologies, they don't have any direction that approaches the efficacy of a transformer
You can have a transformer based index built on top of a graph database. Much of the explainability tech (people who prefer "alignment" also prefer "mechanistic interpretability") is reverse engineering the real world graph that's hiding in the weights.
https://research.google/pubs/the-case-for-learned-index-stru...
That's literally not useful for the sort of applications people want to use transformers for.