Comment by ericmcer
9 days ago
Is referring to these new models as "LLM"s still correct anymore? The frontier models are a more complex orchestration of LLMs and many other programming techniques, they aren't just a weight set anymore.
The whole purpose of this is to let non-technical users say things like "make a flappy bird games" and have it succeed after using 10 million tokens. A decent engineer could build the same thing in 20 mins with a small local LLM.
Same thing with a cyberattack, someone who generally knew what they were doing could quickly surface the info to perform each step of an attack. The govt just wants to ensure random 12 year olds can't type: "how I hack fortnite?" and get a step by step guide.
I think it correct to refer to the models as LLMs. The references I can find for the techniques used seem to boil down to transformers + optimizations to make transformer-index larger/faster + chains-of-thought-optimization/the ability to add real time and other data to the context window.
See: https://kenhuangus.substack.com/p/frontier-ai-models-technic...