Comment by famouswaffles

Even logical reasoning we take for granted, like variable substitution, is difficult for an LLM.

https://paperswithcode.com/paper/the-reversal-curse-llms-tra...

This was a retrieval from training issue, not an inference one. and people have this issue too.

Variable substitution is just not a thing for natural language. sometimes it rings true, most of the time it makes no sense to assume it.