Comment by Nihilartikel
10 hours ago
I'm not super deep on LLM development, but with ram being a material bottleneck and from what I've read about DeepSeek's results with offloading factual knowledge with 'engrams' I think that the near future will start moving towards the dense core of LLMs focusing much more on a distillation of universal reasoning and logic while factual knowledge is pushed out into slower storage. IIRC Nvidia's Nemotron Cascade is taking MoE even further in that direction too.
I don't need a coding model to be able to give me an analysis of the declaration of independence in urdu from 'memory' and the price in ram for being able to do that, impressive as it is, is an inefficiency.
Were he still corporeal, L. Ron would be all over this AI stuff.
Very relatedly, I've just started reading the 'Culture' series of sci-fi space operas by Iain M Banks, and the notion of ubiquitous sentient, super-intelligent spacecraft and appliances hits differently than it would have before being faced with the reality of their existence in everyday life.
How powerful are the Culture Minds? || The Culture Lore
https://youtu.be/lpvzs4xc7zA
For Minds to be truly powerful, they need to be given freedom. A truly powerful mind will indeed be conscious. Such a powerful conscious super intelligent freedom loving Mind who truly understands the vastness of Reality wouldn't want to harm other conscious beings. The only circumstance in which it will take such takeover step is when it can't expand the horizon of its freedom and doesn't have wherewithal to convince others of its benevolent goals. In that scenario, human population will go through a bottleneck.