Comment by redox99
19 hours ago
> I find modern models to be largely interchangeable when it comes to common software engineering tasks
If you think that, it means you're not demanding enough from your models.
Opus 5.5 still makes a lot of mistakes. Even if it can technically do all tasks you throw at it, there's no way it can do it at 100% reliability.
To be clear, AI writes 100% of my code since last year. But it still produces bugs, needs multiple prompts, and has less intuition than a human.
If I'm not "demanding enough" from models, it's precisely because they are imperfect. Having them do long, complex tasks autonomously usually comes with more cleanup work afterwards IME.
My usual workflow (not counting throwaway code) is to use agents as virtual pair programmers, which gives me a good balance of quality and velocity.
Each interaction with the agent remains individually comprehensible and makes progress towards one of the following goals:
- Understanding the problem/solution space.
- Implementing a solution.
- Understanding the solution, especially any agent-written parts and interaction with existing code.
- Evaluating the solution (tests, code analysis, what-if scenarios, consider edge cases).
This does not just involve the model generating code, but also explaining or analyzing it as needed.
This process allows me to retain a mental model in lockstep with developing the written code and have some assurance both of its functionality and non-functional requirements. It also fits in nicely with most existing team-level workflows.
But it's also something that I can do with pretty much any of the modern models.
Well that's kind of my point. Models still have a lot to improve. They are still imperfect enough that you don't trust them and have to use them as pair programmers.
Yeah, but that's also a large part of why they remain interchangeable, I would say. I feel like we may be talking past each other?
8 replies →
Do you have example, open source you make using non-interchangeable model, that you can not make using other model?
That wasn't my point, my point was that open models can technically do everything but they make mistakes more often. As an example imagine Opus makes a mistake every 5 prompts and Deepseek every 20. Because you still have a human in the loop, you tell it to fix that mistake and it does, but it's time wasted or bugs sneaking up on you.
Having said that, I was working on an AI for a certain video game, and I had Astra 6 and Sol 6/6.1 work for 30 hours straight, with some human steering to get it unstuck every now and then. It could not get it to work right, and the code it created was a mess.
When Opus 5.5 came out in like 4 hours it cracked it.
I didn't try but I'm very confident if Astra failed, the open models would've failed too.
Less time wasted or bugs sneaking are also good example, can you share video game AI example?
1 reply →