Comment by lnwlebjel
2 days ago
Thanks for posting this, it's a very interesting case study. Considering that the thing they seem to excel at is this type of writing, it's interesting that they still seem to be only ok at it if you're trying to produce a serious, genuinely useful output. This fits with my experience, though yours is much more extensive and thorough. In particular I fully concur with the voice/tone, and the need to verify everything (always the case anyway), and "Never abdicate your role as the human mind in charge" -- sometimes the suggestions it makes are just not that good.
Question is, do you think this process was faster using the various LLMs? Could two (or N) sufficiently motivated people produce the same thing in the same time? (and if so, what is N). I'm wondering if the caveats and limitations end up costing as much time as they save. Maybe you're 2x faster, if so that would be significant and good to know.
In the abstract, this is similar to my experience with AI produced code. Except for very simple, contained code, you ultimately, need to read and understand it well enough to make sure that it's doing all the things that you want and not producing bugs. I'm not sure this saves me much time.
I think it was faster in that I would have never written the book without the LLMs. Essentially they unlocked the swirl of thoughts and notes that lived somewhere between my head, TextEdit, emails to myself, and anywhere else I stashed things.
It's like it unblocked the "hard part" (getting the words into a coherent form for others), while letting me focus on the "value parts" (my unique perspective / ideas).
It might not be that overall it saved me time, but it made it a hell of a lot more fun, so in the end I completed it - and maybe AI helping us see things through to completion is where we'll see a big unblock in human potential.