Comment by mikepurvis
10 hours ago
But isn't that really just about giving "front end" models more access to specialized tool libraries, which include models tuned to specific tasks? Like the first model says ah, we're being asked to code something, oh and we've been provided with some example code, let me invoke a tool call to my model the recognizes many languages, that model says that we're looking at ocaml. Okay, I better pass this off to my ocaml model which will decipher the supplied code and make a plan for what we do about the user's intent. The ocaml model recognizes that there are tests in the supplied code, let's have the special testing model have a look at the testing strategy and see how that fits in with what we just implemented, etc etc.
And perhaps at the end it all gets a single pass by a god-tier model for overall sanity and congruence, but the actual work, planning, coordination, and even user interaction was done by cheaper and faster agents of much more limited capability.
It’s kinda funny that your last paragraph is basically describing why sparse files, sparse matrices, etc. are used in other contexts.
It really is absurd to ask programming questions to a model also trained about the lifecycle of a fruit fly.
Instead of building small models from scratch, we train an enormous model and use ridiculous amounts of GPU memory. In the end, the whole thing is shoved into RAM because we don’t know where the useful parts are…
We certainly would know where they were if they were just in smaller models in the first place!