Comment by glimshe
4 hours ago
This was a good example where a talented engineer worked hand in hand with AI to get the decomp to work.
I think some people are brute forcing it through a lot of agent time and expense, but they used 6.1 Sol Medium, a great but not a top capability AI model+effort.
This matches what I've been hearing regarding talent and experience making a big difference in task speed and cost when using AI. When prices stop being subsidized, this will be a key factor when evaluating engineer performance: "you've used a lot of tokens and don't have much to show for!"
Who knows where this is going to end, but Talent+AI still beats Vibe work by a lot.
I tend to agree with this perspective however I'd like to point out that using token quantity and cost as a measure of productivity is counter-intuitive. The variance for a given task can be extremely high regardless of the input, not to say that there isn't any skill to prompting or that talent doesn't matter.
Ultimately it feels pretty similar to using lines of code as a measure.
Instead I recommend setting a budget per head and evaluating performance traditionally, allowing for exceptions in exceptional circumstances of course.