Comment by WorldMaker

I think that LLM's are a broken mirror or maybe disco ball. What you get back is almost always a reflection of what you put in, just garbled through and with the reflections of so many other people. The best parts that I often get back are mostly just the stuff that I already put in it.

I think that's also where LLMs get the most dangerous. That ELIZA effect of telling you exactly what you told it but in a slightly rephrased way makes you think that you think that your ideas are genius. You've got someone "actively listening" and telling you whatever idea you just gave it was the best idea it ever heard.

That's not what you want from a serious writing partner. You want a writing partner to challenge you, to point out your mistakes and flaws, to make you better, to offer a truly different perspective to your own. You don't want a dark reflection from a "yes man" to spiral into.