Comment by elliot07
5 months ago
The cost is absurd (compared to other LLM providers these days). I asked 3 questions and the cost was ~0.77c.
I do like how this is implemented as a bash tool and not an editor replacement though. Never leaving Vim! :P
That 0.77 can save hours of work though, fighting with or being misdirected by other LLM. And, relative to hourly rate, or a cup of coffee, it's incredibly insignificant, if just used for the heavy questions.
My LLM client can switch between whatever models, mid conversation. So I'll have a question or two in the more expensive, then drop down to the cheaper for explanations/questions that help me understand. Rewind time, then hit the more expensive models with relevant prompts.
At the edges, it really ends up being "this is the only model that can do this".
After playing with it for a few days, I agree the cost is easily worth the $$. It's almost in a different playing field of "normal" LLM usage.. and more so in the "good productivity tools" realm.
Yep, my experience as well. It's just not worth it.
It burns through tokens like crazy on a small code base https://i.imgur.com/16GCxiy.png