Comment by swatcoder
11 hours ago
Yes.
The infancy phase of this technology is represented by the pursuit of making wildly grand, wildly expensive, all-purpose models that somehow discern a user's full accurate intent from a lazy, underdeveloped, vague idea that they ambiguously and poorly express in a couple dozen words.
The adolescence will arrive as those outsized and ill-considered ambitions collapse and we instead see a cambrian explosion of restrained but efficient model+harness-tuples that have been distilled, finetuned, and rigged to deliver on narrowly scoped but idiosyncratically-shaped tasks with incredible efficiency and erogonomics.
At work i only had early copilot which was hysterically bad at everything. As i wanted it to do the same task repeatedly and could spot wrong results instantly i kept evolving a prompt that attempted to correct all ways it found to do it wrong. It kept inventing new ways to get it wrong until it eventually got it right 90% of the time. My theory is that an avanced model that has no issues with a task could do the prompt enginering much better than i ever could. You could for example run x different queries that all do the same thing and compare the results y times. If there are >1 correct results and the wrong versions are all unique you should be able to drill down to a valid result with even a truly shit model running on a potato. Basically what humans do.
Sounds like mainframes. But then, these never quite disappeared to the date.
This idea has failed to pan out time and time again - people have an instinct that hand-crafted finely-tuned specialized AI systems must be optimal, but throwing more scale and compute to something more generally smart always wins out. It's especially palpable just looking at the last few years of LLM's: a frontier model with all the world knowledge you can stuff in it and every tool at its disposal has always performed the best at all tasks. Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence.
https://news.ycombinator.com/item?id=49322695#49323341
The Bitter Lesson is very popular right now. It seems true right now. It’s having its moment right now. That doesn’t actually mean it’s axiomatically true.
Commenter below gets it absolutely correct: stockfish, which runs on your 5 year old phone, is dramatically better at chess than Fable. Like, so much better that it’s not even remotely comparable. The theory of the Bitter Lesson, and it’s only a theory, is that LLMs could eventually outperform stockfish. It’s not true today and it remains to be seen whether it will ever be true. For now, specialized models are absolutely better at specialized tasks.
This seems really backwards. The Bitter Lesson is all about large data-based approaches vs hand-crafted ones, it doesn't say anything about language models not trained specifically for chess.
I can't find the comment you're referring to, but the latest versions of stockfish are based on neural networks trained on millions of games, so if anything the Bitter Lesson turned out true here.
16 replies →
But isn't that really just about giving "front end" models more access to specialized tool libraries, which include models tuned to specific tasks? Like the first model says ah, we're being asked to code something, oh and we've been provided with some example code, let me invoke a tool call to my model the recognizes many languages, that model says that we're looking at ocaml. Okay, I better pass this off to my ocaml model which will decipher the supplied code and make a plan for what we do about the user's intent. The ocaml model recognizes that there are tests in the supplied code, let's have the special testing model have a look at the testing strategy and see how that fits in with what we just implemented, etc etc.
And perhaps at the end it all gets a single pass by a god-tier model for overall sanity and congruence, but the actual work, planning, coordination, and even user interaction was done by cheaper and faster agents of much more limited capability.
1 reply →
Good point.
Dumb AIs are needed for customer service. Most of that industry is still at "press 1 for sales, 2 for billing..." and needs something that will run locally on a 1U server.
5 replies →
Yes, but an LLM will just call stockfish if it needs to play chess … sure if you arbitrarily constrain an LLM to use no tools it’ll suck at chess. But no one is using LLMs in isolation. Even consumer-grade, bone-stock ChatGPT has tools.
3 replies →
Maybe depends on how you ask it? Directly, or let it write a chess program? I think the latter can yield way better results.
No. The bitter lesson is about capabilities. GP is talking about efficiency.
GP isn’t suggesting that focused narrow model(s) will be more capable than large model, but that many small focused models can have sufficient capability while being more optimal.
Also, the bitter lesson is just wrong. The bitter lesson is about hand tuned AI vs computational general methods. However in truth today’s AI uses both. We have general compute heavy models which require narrow expert instructions (eg tools internet docs).
LLMs would not be as good without expertly written context, and expert context without LLMs aren’t as good either.
> The bitter lesson is about hand tuned AI vs computational general methods. However in truth today’s AI uses both. We have general compute heavy models which require narrow expert instructions
The models are not even really trained bitter lesson-style anymore. That concept peaked during the era of pre-train scaling, back when it was thought that making a bigger and bigger GPT-3 would automatically solve all problems through prompting. In 2026, the most important part of training is post-training, which uses vast quantities of niche, hand-curated data to fit the models for specific tasks in domains like tax law.
It’s not hard to fine tune a 4B parameter model to outperform GPT-5.x or Claude on a domain-specific task. You don’t even need GRPO or whatever the flavor of the month RL technique is either. I suspect what you’re saying is that massive frontier models are the best for doing big slow tasks on human time horizons—and I emphatically agree—but keep in mind LLMs are useful for quite a bit more than just multi-turn chatbots; and in those instances, throwing infinite compute and latency budgets at a problem is generally not a great way to find the “best” solution :)
I'd respectfully push back on the framing here.
If you look at value as purely the LLM output, then there's a valid argument that the best frontier models will always be better than fine tuned specialists. (I'm not convinced personally, but it's a defensible claim)
But that misses two dimensions: 1. The cost of acquiring that output 2. What is actually "good enough" for that specialist domain
Not every output needs to be the best to produce value.
And as specialist models increase in cost, their cost/value proposition goes down.
At some point, there's a threshold where cheaper, fine tuned models are "good enough" at the task and also substantially cheaper than the expert models.
That's where fine tuning helps.
Personally, I became a believer in fine tuning after fine tuning a 1B Qwen model as a second pass over my local voice transcription app, achieving excellent accuracy at ~zero token cost and waaaay lower latency than if I'd invoked my Claude subscription under the hood.
At some point, the idea of cost/benefit analysis in the software business turned into the benefit analysis. The amount of money going into the frontier LLM model game is fantastically ridiculous. Being much better than the free resources doesn’t even touch how much better they will have to be to justify the expense of creating them, let alone continually maintaining these services.
> It's especially palpable just looking at the last few years of LLM's: a frontier model with all the world knowledge you can stuff in it and every tool at its disposal has always performed the best at all tasks. Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence.
Absolutely false. At least when it comes to multimodal inputs, even a simple classifier will outperform the largest LLMs who still hallucinate details or don’t describe audio and images accurately.
And there’s also the issue of cost/inference speed. Running a trillion parameter model for all tasks will be incredibly costly, require a cloud API, while a tiny CNN can be run locally or at a cost multiple orders of magnitude lower.
This idea has not failed to pan out at all. I work for a startup that is exactly what GP described, and am set for life because of how wildly successful it is. Notably, we are successful, in a genuine sense of the word: we bootstrapped from running tiny models to larger and larger models on our own slowly improving fleet of GPUs, and now have millions in revenue without a single dime of outside investment. Conversely, you cannot call taking on ~1 trillion in debt and purchase commitments to scale "success". OpenAI and Anthropic are underwater financially. To be precise, they're in the Mariana Trench.
Wait, you actually found a viable counter to The Bitter Lesson? Please say more
11 replies →
Do you have a website?
Are you willing to say the same of the startup or are you still stealth?
Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence.
VibeThinker 3B constitutes extraordinary evidence, IMO. The first such evidence I've seen myself. Very small model, very low literacy, almost no world knowledge, but it is as good at math and logical reasoning as models a hundred times larger.
The Bitter Lesson is a valid and trenchant observation about how about we got here, but I think it's a mistake to assume it tells us very much about where we're going. Too much has changed recently and is still doing so.
So theoretically, if you give that model the means to find information, ascertain the quality of said information, it could still reason its way to an proper answer?
Is this whole thing than maybe a read vs write optimisation again? Spent more time and effort training more knowledge into the model upfront and get it out in a single question instead of training a small model and needing more steps to answer the same question?
> VibeThinker 3B constitutes extraordinary evidence.. math and logical reasoning
Any similar model aimed at coding?
A >10B model for mass spawning/swarming and reporting back to a larger model
1 reply →
Someone will eventually figure out how to package it all into a single, cheap chip
That you can then write text to program and make applications with.