← Back to context

Comment by IshKebab

2 hours ago

Yeah enjoy it while it lasts. This is basically where LLMs were with coding a year or two ago. Give it a couple more years...

I just started using opus 4.8 and it's still trash at very domain specific stuff and debugging. It went down a rabbit hole trying to coerce the algorithm to work for 1 data point, which dramatically overfit the code. Later it did 11 experiments to try and figure out how to do the math, each one yielding an accuracy no better than a coin flip and each one it asserted beforehand that it had it this time, 100% accuracy.

We've got some time.