Comment by the_snooze

I agree. There are ways not to slide down the slippery slope. Open-sourcing the project at step 2 sidesteps the whole thing. Or being transparent and realistic about practical limitations.

But in large part it is the psychology of being an "ML/AI startup" that is the trap — thinking SaaS is about the software and not about the service. Then everything else is secondary to the holy algorithm. Manual human labor is seen as just a stopgap measure until the automation is perfected, and to acknowledge that at all is tantamount to admitting imperfection, and thus failure.

Theranos is an excellent non-software example. Presumably at some point in her life, Holmes really did want to make blood tests more convenient for patients. But Theranos' eventual obsession with the Edison device made them willing to sacrifice more and more on its altar — money, credibility, patients' safety — until it destroyed them utterly.

The difficulty is that in a VC world, you admit that you will more-or-less permanently need humans in the loop, which kills margins and scalability. At Google/Meta the number of SERP raters and content moderators are only a few tens of thousands, serving a population of billions - and this only after major success. But at Uber or DoorDash, every sale requires a human in the loop from the start. It’s better to be in the first category than the second. As of now AI startups are seen more to be in the first “pure software” category, meaning the margins are expected to be sky high. Of course, the risk here is hardware costs (which will likely come down within the decade for reasonably useful and general models), and humans in the loop (which likely will be a permanent fixture given the hallucinations and opaqueness of the current generation of LLMs).

Re: between steps 3 and 4:

"That's pride [reputation] effing with ya!"

Reputation is everything.