Comment by daxfohl
10 months ago
So is the real engineering work in the agents rather than in the LLM itself then? Or do they have to be paired together correctly? How do you go about choosing an LLM/agent pair efficiently?
10 months ago
So is the real engineering work in the agents rather than in the LLM itself then? Or do they have to be paired together correctly? How do you go about choosing an LLM/agent pair efficiently?
> How do you go about choosing an LLM/agent pair efficiently?
I googled "how do I use ai with VS: Code" and it pointed me at Cline. I've then swapped between their various backends, and just played around with it. I'm still far too new to this to have strong options about LLM/agent pairs, or even largely between which LLMs, other than "the free ChatGPT agent was far worse than the $20/month one at the task I threw it at." As in, choosing worse algorithms that are less idiomatic for the exact same task.
I also wonder how hard it would be to create your own agent that remembers your preferences and other stuff that you can make sure stays in the LLM context.
...Maybe a good first LLM assisted project.
No need to write your own whole thing (though it is a good exercise) — the existing tools all support ways of customizing the prompting with preferences and conventions, whether globally or per-project.
The Aider leaderboards are quite helpful, performance there seems to match people's subjective experience pretty well.
https://aider.chat/docs/leaderboards/
Regarding choosing a tool, they're pretty lightweight to try and they're all converging in structure and capabilities anyway.