Comment by XCSme
18 hours ago
I think it's the opposite, with the right guideance, testing, frameworks, and using the top models today, implementing something with AI is most of the times better than what most developers would do.
I basically only manually test e2e myself, other tests are automated, code review is automated. I can tell the model to test for me too specific things or to add tests for specific potential issues, performance benchmarks, compared different implementations, etc.
The focus is a lot more around the code than on the code.
2 things.
1. If what you say is working, you have a working software factory that should be capable of matching the output of dozens of engineers.
What very impressive externally verifiable results have you had with this?
2. If you are working in software with plenty of customers, my strong suspicion is that there are people on your team who are looking at the code who furiously trying to reign in your output.
I make my own software, so no team to look over the code or be bothered by it. I don't even know how "vibe-coding" works in a team environment, because for me now it feels like it's "ideas to app" directly, so dumping by brain/ideas directly into a functional product.
I think this one is really cool[0], will be a free piano learning app. I do have other projects, but they are all at around 80% too, because some systems are shared amongst the projects and have to be finalized too (i.e. now I'm implementing my own transactional/marketing email service on top of Amazon SES, I need it before releasing ultimidi so people can register and receive email confirmations).
[0]: https://game.ultimidi.com
That is a cool app. But if you’d said you’d made it by hand in a few months I would have believed you.
If I had a working software factory like you describe, I’d expect you to have hundreds of apps of that level of complexity in a year.
> I don't even know how "vibe-coding" works in a team environment, because for me now it feels like it's "ideas to app" directly, so dumping by brain/ideas directly into a functional product.
But you don’t have any users much less paying users, so you have no idea if this system works when you do.
3 replies →
> What very impressive externally verifiable results have you had with this?
I notice this weird hostility whenever the topic of AI coding comes up and it's never made much sense to me. If someone told me about their new method for practicing guitar I'd feel like a real tool if I demanded they prove it for me then and there.
People don't owe you their "very impressive externally verifiable results" - /u/XCSme already posted their app in another comment, it looked fine to me.
You've made your ideological position very clear here, you don't need to keep heaping it on.
More like, if someone told you their workout program that takes 5 minutes a day lets them lift the same weights as an average Olympic weightlifter, you'd ask to see the results.
I don’t care what somebody believes about the pile of code they’ve got sitting on their own computer. The appropriate level of engagement is sort of: well, they don’t owe us any evidence we don’t owe them any credulity, and if we’re all happy to ignore each other that’s fine.
I can see why some folks here want to quibble, though. In other comments in this thread, they’ve compared to code quality favorably to the output of an average developer. That’s slightly insulting to the field in general (although, I guess most programmers have a low opinion of average code quality).
If someone tells us about their new method for practicing guitar, it is very natural and not at all a toolish behaviour to hand them a guitar and say “go on, play us something”.
In fact it’s actually a very socially agreeable action, as it gives them the opportunity to show off their new skills without them looking like they’re bragging.
Now, if you happen to know for a fact that the guy cannot actually play guitar, then you’re just setting him up to embarrass himself, which is maybe an extreme punishment for the relatively minor crime of spouting some bullshit. I could buy that that is hostile, sure.
> with the right guideance, testing, framework
But you didn't write or read any of the tests so how do you know they are accurate?
You don’t, that’s why in agentic world your codebase is only as good as what you can prove. For this reason, you’re going to see more languages evolving feature like capability permission, effect/coeffect types, refinement types, formal verifiers, strict type checkers, static analysis and so forth. Tests are only a small part of the verification. These ideas are old and have sat outside of the mainstream coding world, but the value proposition in the ai age has changed enough their relevance is renewed.
Not arguing for not reading test code. A lot can be achieved by instructing agents to balance out the testing pyramid with the right amount of fast end to end tests, property based tests for the right things, parametrized example based tests. Ensuring the local and CI has the right mix of tests running at right time. On projects where I have less time to review AI generated code I channel my anxiety into setting up guardrails and processes for the agents and then force them to bump up against them. In the end I view it as creating frameworks which allow me to outsource some of my attention to the agents, so that I can claw back some of that time to go set up more guardrails for more agents who are working on something else.