Comment by mr_roboto
19 hours ago
He'll pull it off, get feature complete, and spend a year or two with better models as they are released ironing it out. We just did an experiment, in four weeks generated 3500 commits, 200k sloc with a 1:1.15 ratio of production to test code and it all works very well. We are well on our way from 10x to 100x.
Did you check with the end user as to how well its actually working? My prior company is doing this and their platform has gone to latency/bug hell.
It is for internal use and so far so good. We are finding bugs and we just point the models at it with a strict red-first TDD approach and they've done very well fixing them. It certainly isn't perfect but the issues are very similar to what you get with a team of humans doing the same thing.
The main takeaway though is we had one person, in a single month, do the equivalent of at least a couple years of work for a small team. We view bugs and performance issues as a currently inevitable part of the process and we are dealing with them with models just like we do with humans on every other project already. Overall though it is representing an absolutely massive speedup. If you work in an environment where the models make egregiously more mistakes than your human counterparts then I am quite jealous.
and if you enable dangerously_skip_permissions you can probably even get to 1000x