Comment by miki123211

I think large-scale systems are optimal when the number of tasks is relatively bounded and costs don't matter.

When you're doing something "at scale", you want smaller models, weaker heuristics and rules written in code, as they're much cheaper, much faster, much more predictable and much easier to reason about.

If your task is "tell me which pages on the internet are most relevant to this search query", running Claude Fable to evaluate every single page would be ridiculous. You may run something like it at the end, to synthesize information from a small result set for example, as that's a "human-sized" task.