← Back to context

Comment by imtringued

5 months ago

The problem with LLMs is that they struggle to generalize out of distribution. By training the model on a sequence of semantically tagged steps, you allow the model to stay in the training distribution for a larger amount of prompts.

I don't think it is 100% a scam, as in, his technique does improve performance, since a lot of the benefits can be replicated by a system prompt, but the wild performance claims are probably completely fabricated.