Comment by biophysboy
21 hours ago
I have recently come to the conclusion that thinking for 2 seconds and using a cheap model with a slightly more detailed prompt works just as well as zero-shotting an idea with a fancy model. I work in science, and instead of asking the model “write a topic extraction algorithm”, I just say “hey look at this matrix factorization script I found in a repo, now make it use plotly and duckdb”. Have others come to the same conclusion here?
It makes me skeptical that the flagship companies are sustainable. Every company is going to maximize “fuel efficiency” to save time and money.
Then again, maybe the cheaper models have more markup for them, in which case they are probably happy w this arrangement. I’d be curious to know how the money making varies by model.
Really? It's worth it to you to spend 10 minutes thinking about how to prompt a dumber model to save $0.05? (not that open source models are dumber any more)
Its not just about saving $0.05. There are many legitimate reasons to not want the AI to do the thinking (aka architecting or planning) for you. In those cases where one-shotting is undesirable or unnecessary, closed frontier AIaaS has no advantages over open weights.
Yes? The difference is often 10X or 100X with very little time lost. I’m learning how to give it enough info pretty quickly.
Edit: I also have to read the methods anyway for scientific accountability/integrity anyways, so I may as well play that role at the outset.
Well its def not $0.05, I just started using claude sonnet 5 and Ive found most simple questions might be 0.05 cents, a unit test is something like 0.10 -> 0.20 and small features and classes get into the individual dollars. Sure its a lot faster but at the end of the day its not cheap.
Plus there is something to say about being in the drivers seat, youll have a much better idea of how it works instead of needing to talk to claude and hope its correct. Since most LLMs also not very good at ideas even in my experience with better models its better to think for 10mins, youll get a much high quality result
Yes, they can’t magically resolve ambiguities in the prompt. Hence it is better to make something clear at the outset