Comment by howunfortunate
1 day ago
> There’s no reason to think this.
There are lots of good reasons to think this.
The first is that LLMs used to be bad at each of these nerd things, then toppled them like dominos. There's a pattern over time.
Another reason to think this is g. Across every known measurement, human (and animal) intelligence is convergent. Being better at one thing correlates with being better at another thing. Although the reason is not perfectly clear, the pattern is well established, and seems to apply to LLMs too - GPT-6 is smarter than GPT-3 at everything, not just math. There's no reason to expect different for GPT-9.
That's just cocktail party level of thinking. LLMs are getting better at math, code and logic (and marginally better at science and general knowledge) because these are domains that can be objectively verified and thus there is a potentially infinite supply of 'facts' to generate and train on. These are very powerful but ultimately very abstract domains. For everything else the messy real world and its physical bottlenecks gets in the way and there's little reason to expect progress to accelerate. It still takes months to get mice to reproduce and run experiments on, no matter how knowledgeable about biology the models have become.
If anything, in some domains frontier models have become worse - claudisms and chatgpt idioms are making them notoriously bad at prose without a considerable amount of prompting and tweaking.
Is good writing verifiable? I don't think it is, but LLM's have been hillclimbing writing quality. That being said this is with the help of RLHF.
However there are many other domains which have verifiable rewards in the process of learning them, despite their overall impact not being verifiable. For example, the life sciences, an LLM could be given access to data about an organism, and then make predictions about how a drug or gene therapy will affect that organism. In economics, LLM's could create models of behavior, evaluate predictions over time and see how well those predictions match reality.
>claudisms and chatgpt idioms are making them notoriously bad at prose without a considerable amount of prompting and tweaking.
I think this is because a lot of people genuinely like the claudisms, even though a small minority of technical people don't.
The quality of natural language output from today’s LLMs seems no better and often worse than LLMs from 18 months ago, at least in my experience. It’s much harder to build a good RL loop for something subjective like good writing, compared to something with an objective and verifiable right or wrong answer, such as software or mathematics.
5 replies →
It's not at all clear that LLMs have gotten significantly (much less hillclimbing) better at writing since GPT4-ish. They did get better at oulipo-like challenges (writing under silly, verifiable constraints like Fable's viral 'facetiously' poem) but that's far from being all there is to good writing.
Anecdotally, creative writing communities do report drastic differences in quality between models (it is my understanding that Gemma 4 is especially praised) so there is something to it being verifiable but it's a far cry from being something that can be distilled into objective math/code-like evals. One could even say it comes down to vibes.
>For example, the life sciences, an LLM could be given access to data about an organism, and then make predictions about how a drug or gene therapy will affect that organism.
Yes, biotech companies are doing this right now but ultimately they are just predictions and you still need cold hard biological data to ground them and iterate on. That's slow and expensive, especially for the juicy fields (human biology, food and crops, clinical trials) and you can't just plug billions of VC capital into the pipeline and hope for RSI. Physical (gotta procure all those labs and their equipment), human (gotta hire and pay specialists to run experiments), biological (gotta wait for organisms to reproduce, drugs to take effect, crops to grow), regulatory (gotta convince agencies that your fancy new drugs are legit) bottlenecks get in the way.
And biology is one of the easier fields where a path to RSI (if not accelerating) is conceivable. Good luck iterating on macroeconomics data where there are no replicates and 'experiments' take literal years if not decades.
>I think this is because a lot of people genuinely like the claudisms, even though a small minority of technical people don't.
My uncharitable take is that the abtruse jargon makes people feel smart for understanding it and gives a sense of belonging (as a closed circle of initiates who understand LLM cant), just like rationalists love to repackage old or unsavoury ideas under new nerdy smart sounding names.
3 replies →
There was a time when people believed in Moore's law too. Now even Jensen Huang says Moore's law is dead. There's also the adage from investing: past performance does not guarantee future returns. A little CYA sure, but also acknowledgement of latent factors and the entropy of the universe and its affect on any predictive modelling one might do.
I don't agree that they get uniformly better at everything, especially subjective skills.
For example, they have become more and more unintelligible when you ask for explanations or descriptive text. They assume you see the same context as them and shortcut explanations.
I've had to craft a skill to get them to produce remotely understandable explanations of even mildly complex/non-mainstream subjects.
This may be Curse of Knowledge https://en.wikipedia.org/wiki/Curse_of_knowledge on their part, and it also impacts human experts but still. Becoming better at one thing does not mean you become better at everything else, although I do agree with you that the better they become the more things there are they become good at, but their ability is still quite jagged and maybe increasingly so.
The delineation between knowledge and intelligence is useful here.
Knowledge can occasionally get in the way of some tasks, such as a master illustrator trying to draw like a child. Or as you've mentioned, an expert trying to explain to a beginner.
Even so, I disagree with you about model progress on communication - maybe there's a little jaggedness between minor model versions, but Opus 5.5 is a much much better communicator than, say, Claude 3 Opus.
I don't agree that Opus 5.5 is necessarily a better communicator than older models. In fact, I ran my own tests, and the difference wasn't very clear, and in terms of personal preference, the writing style of newer Anthropic models has really let me down. It doesn't reason like a person that imagines what the person they're explaining to might know or not know, or what they might think, as one usually does when trying to explain something to someone. They just fact-dump, like the problem solvers they are, since they are trained to solve user's problems correctly, efficiently and comprehensively, but they probably don't have a good internal representation of the user they're talking to, they are not really trained to imagine what we may think or feel, and thus, in my opinion, make pretty bad communicators (cf. the ocean of vibe coded READMEs on github that are horrendous to read).
For example, when asked to write a report on Docker and explain how it works, one of the first paragraphs was : "1. *Docker Engine*: The core component, which is a client-server application that uses a server component (the daemon) to manage containers and a client interface to interact with the daemon."
This is perfectly correct and makes sense for someone that is familiar with software engineering and already knows what containers are. Not so much for someone wanting a introduction to it. Do you imagine writing something like this at the very beginning of your explanation of Docker ? It even said: "I wrote it in the neutral register from your publication preferences, with no em dashes, and defined terms on first use.", when it didn't even define what a container is. I chose Docker at random, because I could judge if its explanation made sense, since I know how it works.
I’m not sure I agree that g explains all model capabilities. I think “claudespeak” is a consequence of the shift from training on RLHF to training on verifiable rewards. Thus the models seem to become worse at storytelling, fiction, readability, not better.
Even if we do accept that models have some “g” and that it correlates with all domains, there’s no reason to think that the current trajectory of LLM intelligence will grow more quickly or even as quickly. In every other technology we have an early period of slow growth during experimentation, and then a rapid period of growth, and then stagnation.
For all we know, 2026 could be the year of stagnation. It’s possible that we will reach finishing returns. If thats true, then it won’t matter how much that task correlates with overall g, because g won’t be increasing that much.
You are conflating intelligence with usefulness.
> Across every known measurement, human (and animal) intelligence is convergent. Being better at one thing correlates with being better at another thing.
LOL. Human (and animal!) intelligence is notoriously jagged. We're all basically idiots except for very narrow areas where we focus.
This is simply untrue. I tried to think of an animal example so I could give a concession, and there just...aren't any.
Smarter just means smarter.
As anyone who's needed to provide IT support to a doctor, lawyer, lauded academic or host of otherwise "smart professionals", can assert, you can be very very smart in one area and very stupid in another.
See for example Paul Framton https://www.theguardian.com/world/2013/mar/30/physicist-mode...
3 replies →
Cats famously have more brainpower than dogs. But dogs understand us humans far better than any cat.
Someone who actually works with real animals (farmers? Ranchers??) would come up with better examples. But I'm just city boy over here who only meets cats and dogs on a regular basis.
2 replies →