Comment by ghshephard

It's a solid question - and to some degree what https://programbench.com/ tries to measure.

Some of the issues (off the top of my head):

- Note - that my "product" was about 3,000 lines of code - so tiny. But https://metr.org/ should give you some insight into the complexity the models are capable of.

- you have to be able to imagine the product. If I have the time, and energy, to imagine what I want - the model will build it. Here is an example of a much better programmer than I and something he wanted built - https://www.boatbomber.com/blog/claude-fable-5

- These are the first drafts. On average - any complex system needs about 10 years and at least 1000 active and enthusiastic about reporting users to really get robust code. Writing if via LLM doesn't (at least so far in my experience) help that much in reducing bugs if you were previously following any semblance of TDD. Lots of bugs in the code - the products you listed above have literally tens of millions of years of user experiences and bug reports that got them to where they are today. No silver bullet yet - just faster, less effort - and it enables non-technical people to create (still buggy) products.