Comment by amelius
3 hours ago
I don't understand. If you have a model that can do bash examples already (your subagents), then why would you need to train a model?
Or are the subagents generating your training data using a closed/paid model?
3 hours ago
I don't understand. If you have a model that can do bash examples already (your subagents), then why would you need to train a model?
Or are the subagents generating your training data using a closed/paid model?
A very small, highly specialized model can use negligible resources (CPU, energy) to accomplish the same task.
For everyday work that happens frequently it's better to have a tiny specialized model instead of making billable API calls or turning your laptop into an 80W space heater for 20 seconds to run a general purpose model.
The large models can be used to generate synthetic training data. Tell them to make up 100,000 tasks paired with the resulting output as a 1-time cost. Then use that to train a small model.
Think of it as distillation, but focused on a specific task.
Given that they're just using it to avoid the googling for bash command syntax, I'm not sure they'll save in the end against the 140k training examples they generated.
Good observation! It would have to be offset with O(140k) queries to the model, which is, well, unlikely.
2 replies →
you can probably generate quite a few example pairs in a single shot, you also likely don't need the best models for this either
This is so cool - I'm aware of this in a vague way. Can you write a little tutorial or give some good links. I want this to be the next new things I do :)
Better yet package it up in a skill!
The models he is using to generate training data are presumably commercial models. He is distilling their bash knowledge into a much smaller model he can run locally fast and cheap.