← Back to context

Comment by imtringued

12 hours ago

Based on the few published subjects, it doesn't look like anyone actually tried to get the secrets.

Usually the way to go in situations like this is to flood the context window.

You will either hit a bug in the context management (sliding window removes the system prompt) or you have diluted the context with so much new information that the attention mechanism stops focusing on the system prompt.

The author also shows that he doesn't understand what batching in the LLM space means, because they conflated the idea of processing multiple emails in one context window as "batching", when that is actually sequential processing. Actual batching would process each email with an independent context window.