Comment by wiremine
7 days ago
It's a good write up, but it's lacking some details, the most important one is: which Claude model was used?
The second issue is: what was tooling and the prompt approach?
(To be clear, I have no problem with the premise of the write up. But without some details like this, it's sort of like saying "I had a bad board on my deck, and my tape measure wasn't able to help me remove the nails. What a bad tape measure."
Opus 4.whatever (it was last week) via a command line interface in the IntelliJ Claude plugin.
The series of prompts weren't particularly interesting or innovative on my part: a paste in of the user report then a few back and forths on fixing it, me reviewing the changes and coming up with the final answer.
It might feel interesting, but it's sort of the crux of the issue. Average or below average prompts will produce average below-average results. The model can't make up for that.
Not saying every problem can or should be solved but AI, but mastery of the tools is kind of important when evaluating the tools. It's like complaining that vi or emacs is slow to use because of the bindings are complicated.
idk i think i'm pretty good w/AI in general, e.g. designed these using it:
https://mtmc.cs.montana.edu
https://bcp.cs.montana.edu
but we can all be better i guess
2 replies →
It's more like asking what editor and keyboard layout they use. Highly relevant to the user but you should simply assume someone describing work is using a setup for it they find productive. If you decide to dismiss their output it wouldn't be over these details.
Not quite. Model is extremely important in the quality of results. Harness can also influence things.