Comment by kennywinker

This is an oversimplification: more data makes models smarter ceteris paribus, but mostly only because auto-regressive training (where most of the general knowledge comes from) is essentially compressing information that can be recalled later if it’s useful (or not recalled). Obviously there are differences in kind within “more data” too, you would much rather have all the books and blog posts in the world than all the fanfiction.

Yes, all three together would be even better. But it wouldn’t be if you had 100x more fanfiction, mostly synthetic, generated during RL to teach a model to be better at writing fan fiction. There are real limits to the amount of knowledge you can cram into fixed-size (downstream of hardware availability) weights. For a period scaling with data was basically “free” because we had the Internet and all the books/media that humans had already created; the data was accessible and limited (at least, the parts we think models should know about) enough and top-hardware big enough that we could basically compress the whole thing.

Post-training/RL are making this obsolete because they’re more about skill/capability acquisition rather than knowledge. They can generate much more data (most of it quotidian/useless, ie an agent made a typo in batch 382829) and clearly seem to cause a kind of mode collapse even in the most advanced frontier models.

We don’t need to make LLMs forget about SpongeBob SquarePants so they learn more about bash. But if I have a question about SpongeBob SquarePants, I don’t need to hear about load bearing seams prefaced with honest caveats after a model writes 400 lines of bash to look up SpongeBob’s family.

And there is probably a lot more SpongeBob knowledge we could put into models if we wanted to: interviews with the creative staff, a SpongeEnv/SpongeHarness modeling how the art/story team work together to create entertaining kids tv, a SpongeBench measuring entertainment value, etc. If a SpongeAgent spends 2000 years in Agent University learning how to Spongemaxx we probably don’t need or want to have it spend another 2000 years writing smoke tests

Thanks for putting this so well. The mathematical evidence for this “general intelligence underlying everything” is the “universal geometry of embeddings” paper. Fascinating read. Or as the ancient philosophers used to say, the one who knows God knows everything.

This makes a ton of sense, and seems like it's coming from somebody who understands LLMs better than I do so I will definitely take your word for it - but it doesn't totally track with my experience of running small local models either. In my experience some models are better at language X and others better at language Y - and all of them are better at language X and Y than language Z-I-just-made-up. I take that to mean there is some relationship between training data and skill. Maybe not enough to make pluggable modules, tho... at least not with the architectures we're currently using.

By pattern matching that SpongeBob episode to the case at hand you provided an example of what you had just explained. Nice. So meta.

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant.

You just defined a liberal arts education.

People keep forgetting that programming is not just about knowing the target programming language, but also an enormous volume of tacit knowledge:

    - Understanding of protocols like HTTP.
    - HTML, JS, CSS, SVG, and everything "web".
    - Understanding of databases, SQL, etc.
    - Abstract code architecture patterns.
    - Understanding the users' requests in English.
    - Responding in English.
    - Command line tool usage (agents/harnesses)
    - Industry-specific knowledge that can be applied.
    - Frameworks, SDKs, applicable libraries.
    - Relevant legal requirements.
    - Etc...

I.e.: If I tell a frontier AI that this project is for a "local council in XYZ location" it can immediately figure out that a scalable, globally distributed architecture is not required. It can also figure out that using local time instead of UTC is not only "fine", but even desired. Or that globalization/localization is not required... or.... required if the council is in some place like Belgium or Canada where multiple languages are officially recognised and supported by the government.

could you share the 3blue1brown videos you're referring to?

This is so right. We training Whisper Large model on 20,000 audio samples specific to a domain and it ended up reducing the ASR by 5% while improving WER of the finetuned domain by 0.5%.

Instead we ended up with no finetuning. We give audio snippet to 2 AsR models, take 3 best transcriptions and ask the LLm to pick the best based on the context. That produced significantly higher accuracy in how an agent understands the users.

You 100% can finetune or adapt/build on top of models, and specialize them or extend their capabilities. That’s literally what post training is.

The problem is that “finetuning” was a 2023 AI FOTM associated with products/demos that were almost exclusively using it for LLM character role-play/output style purposes (ie not in actual systems where they served a more functional role).

This made people think you could train models without replay/real evals by yoloing it with SFT (this is partially an artifact of that era being much heavier on autoregressive training and not so much evals). You really can finetune and get results but you have to treat it like a small ML training run, with real evals, and more intentionality than just “more examples”.

You can find pretrained and -instruct models on huggingface that clearly demonstrate what specialization/staged training runs do.

I’d be very wary of conflating finetuning with specialization/extending a model’s capabilities in general.

Unless you are building a factory assembly line where a model is literally doing the same thing over and over, you almost always want a general purpose model over a specialized one.

Turns out the world is made of simple, specialist processes, not generalists trying to achieve them. Adaptability may be of great benefit in evolutionary terms or for a walking anthropoid, but the majority of biology, chemistry, and mathematics rely upon specialist process for good reason. See also the old trope about robotics: that's what you call it before it works, otherwise it'd be a dishwasher.

The upshot is: use a generalist to create a simple solution once, and scale that. Don't deploy the generalist at scale, that's a waste of resources and an inefficient solution.

Sure but I bet this doesn't translate as well in this way to things like Erlang

<THAT'S A BINGO>

ie what everyone asking for this fails to immediately realize.

But the understanding of the language library ecosystem, or even better, your codebase, could let it execute faster and with less context usage.

> This is roughly what multi-agent systems are built for.

I think I disagree. For some things, maybe that works - but think of a multi-agent system where one agent understands the code, and passes it off to the reasoning agent to figure out what the bug is. This system is going to suck. Because encoding enough info to figure out what the bug is would just be dumping every single line of the code.

So say agent 1 (reasoning) asks agent 2 (swift) to explain what is happening in File.swift. Anything agent 2 passes to agent 1 short of the entire code is a lossy transfer - and then the bug gets missed.

everything is a token

> Sounds like unix philosophy.

https://linux.die.net/man/1/ls

https://linux.die.net/man/1/find

https://linux.die.net/man/1/bash

I want to pick it myself because I want to run this stuff locally on hardware I can afford today. But most people seem happy enough using the cloud, where this kind of architecture could be seamless if it existed. I.e. the query to `/chat/completions` contains an extra parameter `domains: ["c++", "swift", "navigation" "gis"]` and then those are the modules that get allocated to this query. When you start a new conversation no domains are set and it hits a generalist model, but the generalist model includes domains so the next query doesn't have to hit the generalist model. The model could even have a tool it could call to rope in new domains if the scope expanded to include other modules.

Would it? AGI was always about the efficiency gains of versatility, not strictly all forms of efficiency

On the otherhand, your own brain probably doesn't use your eyes to hear.

But I am not asking for a faster horse. I am asking for a draft horse instead of a race horse. I know the tasks I have on hand, I know the VRAM and compute budget I have to run them. I am not asking for AGI, I just want something to edit my little text files.

I would, once it is possible, specifically unplug the shitcoder expert that sometimes shows up and puts `as unknown as` and bypasses the commit hook that is supposed to enforce it. That should be somewhere near java coder expert in the whatever dimensional space. I can feel it being true.

My motivation is somehow the model should be able to work without hitting an external resource every single time and somehow have all the knowledge necessary locally so we don't get rate limited but how can you stay up to date with the latest information while still remaining a good boy scout by not reaching out to tool use and scraping every single GitHub repo and issue every single time?

Except assuming a fixed budget of parameters, there is clearly stuff that's better for programming than others. E.g. Qwen 27b is a better coding model than Gemma 4 31B. More params doesn't automatically win. Perhaps what makes a good swift programming model is a ton of python training, so those two things can't be separated - but that doesn't negate the idea of loading a model that's good at the specific task you want - or has specific knowledge of the libraries and tools for the language you're using at the expense of the ones you're not using.

No, that’s a common misconception. MoE despite the name doesn’t have an “expert” slice that’s an expert in any particular thing. It’s just trained models such that a subset of all memory weights is active on a single token. Often, the weights active on the next token are different. It’s used to improve memory bandwidth (throughput) and also to even out load in a distributed serving model - one GPU hosts one “expert” and the conversation is routed to it for the next token. This requires a lot of movement of the KV cache of course so it’s difficult to retain speed across multi node splits - usually it’s a rack of GPUs that you do this for.

MoE's in the abstract often get presented as if theres explicit layers of experts for any given domain of knowledge, like your coding tasks are being routed to coding experts, but it's really not that at all.

THe original MoE paper from Noam Shazeer et al. is worth a read on this bit, though the paper is admittedly pretty dense. But TL;DR is that each expert layer is learning highly abstract, localized structural and syntactic patterns in the data to minimize the loss function, and its doing this token-by-token (which in some cases may have some domain clustering, but that's just incidental).

When you start batching your queries, even if they all seem like theyre in a single domain, if you visualized the activations you'd notice that most if not all of the network is lighting up on the batched forward pass.

...kennywinker is a 0.001B MoE model, with 2 active parameters.

I mean, I get pretty damn good general reasoning out of Qwen3.8-27B quantized to 4bits, and that knows Swift, Python, Node, Ruby, Rust, etc. etc. etc.

And so far even the biggest model doesn't seem to have a working Make No Mistakes module, so maybe that's not needed

I'm not actually describing a specific how, I'm describing a goal. The goal is models that are made for specific work, not general work. My model doesn't need to know a single line of python, but if it's a coding model perhaps giving it some python is the best way to make it smarter. Ok. Great - give it python! But first maybe check if giving it C++ is actually better than giving it python. Because for my purposes it doesn't need to know C++ or Python, and I have a fixed parameter budget. Find me the ratio of Swift:C++:Python that gets me the best Swift coding agent in my parameter budget, and then train that.

And then ideally, make it pluggable so I can pick what I want from off the shelf components, but if that's not possible - then just train up as many variants as you can so we can all pick the best variant for our current need.

The future will be "AI operator license #123..., subclass 'Primitive Coding'", with your TouchID / FaceID enabled only. And I am not being sarcstic.

I don't think so?

Definitely not MCP, as that pulls info into the context. Unless contexts become REALLY big so that I can add 10B in swift knowledge, that's not gonna help me.

Possible RAG? I don't know enough about how that works, but I think that's not quite it either. I don't want to import facts like "the swift standard library contains a reverse array function", i more want to import knowledge - e.g. the parameters used to generate the text to reverse an array in swift.

Tool calling wouldn't do it either. You'd have to encode every single possible bit of useful info into the tool call, and the tool response would have to encode every piece as well (variable names, function scopes, types defined in other files, etc). E.g. how does it find a bug, if you have to pass understanding back and forth between the brain that understands debugging and the brain that understands THIS code?

I don’t think so because those both live in the context window and as such pollute it when they’re not performing optimally.

I think having unused or rarely used weights doesn’t influence the results as poorly as RAG injecting irrelevant facts.

It sounds to me like some sort of “dynamic MoE” where you can add/create or remove experts on the fly.

I think what you’re describing is the closest approximation we reasonably have right now though.

Only if you fundamentally misunderstand what I am describing