Comment by xmonkee
2 days ago
This is just very clever marketing for what is obviously just a cost saving measure. Why say we are implementing a way to cut off useless idiots from burning up our GPUs when you can throw out some mumbo jumbo that will get AI cultists foaming at the mouth.
It's obviously not a cost-saving measure? The article clearly cites that you can just start another conversation.
The new conversation would not carry the context over. The longer you chat, the more you fill the context window, and the more compute is needed for every new message to regenerate the state based on all the already-generated tokens (this can be cached, but it's hard to ensure cache hits reliably when you're serving a lot of customers - that cached state is very large).
So, while I doubt that's the primary motivation for Anthropic even so, but they probably will save some money.