Comment by spijdar

4 hours ago

It's that, yeah, but coming from multiple orders of abstraction.

In this case, part of the experiment was to see what "oh-my-pi", a "fat and feature rich" LLM harness, could do when coupled with modern GPT, given a 6k~ word overview of a story, and told to come up with a plan to write/review/audit it, making use of subagents and all the fun new groovy LLMisms...

Part of the problem was just "it was basing its style off the last scene/chapter", but part of it was also that its instructions were constantly being "compressed" through repeated compactions. Even with the use of subagents, the "top level" agent's prompt was getting muddied, and in the "review" phase, it began to focus more and more on creating increasingly complex ledgers.

You can see this happen in the "plan" files it created for each chapter, looking at word count:

   1304 d1-ch-01.md
   3701 d1-ch-02.md
   5151 d1-ch-03.md
   6462 d1-ch-04.md
   9587 d1-ch-05.md
  10605 d1-ch-06.md

So it wasn't just that the prose was being based on an increasingly compressed "style" of the prior context window, but the planning for writing each scene was, itself, becoming fixated on the "continuity error correction" process itself, to the point where by the end, it had mostly forgotten about the prose part, and was completely fixated on ensuring maximum state continuity.

This could definitely be fixed, but honestly, I've about had my fill of the "autonomous writing agent" goal. The idea was to make a model that could generate sufficiently interesting stories based on "vague premises" for my personal entertainment, but, "surprise", getting LLMs to actually produce both "new" and "coherent" content beyond what you specify is _hard_.

It seems like you really do need to just stay "in-the-loop" with every scene, and constantly provide correction/feedback, to correct the "semantic drift".