Comment by PotatoFarmsKing
6 days ago
Before LLMs the tech groups I followed were ripping with discussions about this and that topic, what to use and when; I believe these discussions sparked the creation of many frameworks and tools out of "this seems like a good idea, wouldn't hurt to implement it". Unfortunately it all resolves around LLMs nowadays and how to make some LLM work some way or another, we don't even discuss the very topics the groups were created to discuss. I fear science is soon to taste the same thing - discussions about LLMs taking place instead of the actual topics that would be discussed otherwise.
Well LLMs are largely useless and people are realizing that.
Raw dog Chat LLMs are pretty worthless. But run an agent with tool invocation and they get scary good. It's amazing how much reasoning is packed into the English language. Provide your model with enough information and it can pull some miracles out of thin air. It's not the "Replace humans" level yet, but you can automate a lot of stuff you wouldn't expect to be able to automate.
What you are saying, if I follow, is that LLMs basically worthless: it turns out that coding is so simple that verifiable rewards can tune weights surprisingly well for that one peculiar task. ('agentic' is fancy word for letting them run what they write - not to put too fine a point on it.)
You've made the most damning remark against Planet LLM I've read.
2 replies →
No its not doing magic. Im impressed when anyone can play a guitar, because I dont know anything about playing a guitar. Someone who's been playing the guitar for years isnt impressed by all guitar players.
This seems the case with many people using llms to write code. They think everything an llm does is magical.
It will never be able to replace humans with two brain cells.
2 replies →
Doesn't make sense to fixate on LLMs and not the actual Transformer/attention foundation. The Transformer/attention architecture is the breakthrough, not LLMs. Especially the RLHF chat paradigm is 100% a byproduct. Which is easy to see when you look at how ChatGPT originally came about.
DeepMind has already has had real impact on science with the same foundational architecture as LLMs, for protein folding. They won a Nobel prize for it.
LLMs with a verification layer work great (code with tests) I know my field quite well and I can one-shot many useful things. I can't trust any of it but I can trust tests and verification tools.
My friend they have dumped hundreds of billions into LLMS.
The ROIC is not gonna look good if they do not somehow make use of the existing assets...
Not an argument for btw, Im just saying. Ultimately the management answer to shareholders who look at return measures such as that.