Comment by littlestymaar
1 day ago
> Yes, you just break the book down by chapters or whatever conveniently fits in the context window to produce summaries such that all of the chapter summaries can fit in one context window.
I've done that a few month ago and in fact doing just this will miss cross-chapter informations (say something is said in chapter 1, that doesn't appears to be important but reveals itself crucial later on, like "Chekhov's gun").
Maybe doing that iteratively several time would solve the problem, I run out of time and didn't try but the straightforward workflow you're describing doesn't work so I think it's fair to say this challenge isn't solve. (It works better with non-fiction though, because the prose is usually drier and straight to the point).
in that case, why not summarize the previous chapters and then include that as context to the next chapter?
That's what I did, but the thing is the LLM has no way to know what details are important in the first chapter before seeing their importance in the later chapters, and so these details usually get discarded by the summarization process.
Which is why you go back and re-summarize a second time given the context of the important details found out in the first pass.
2 replies →