← Back to context

Comment by int_19h

1 day ago

The new conversation would not carry the context over. The longer you chat, the more you fill the context window, and the more compute is needed for every new message to regenerate the state based on all the already-generated tokens (this can be cached, but it's hard to ensure cache hits reliably when you're serving a lot of customers - that cached state is very large).

So, while I doubt that's the primary motivation for Anthropic even so, but they probably will save some money.