Comment by podgorniy

> Why? If it’s out of domain we know it’ll fail.

To see if LLMs adhere to logic or observed "logical" responses are rather reproduction of patterns.

I personally enjoy this idea of isolation "logic" from "pattern" and seeing if "logic" will manifest in LLM "thinking" about in "non-patternized" domain.

--

Also it's never bad give proves to public that "thinking" (like "intelligence") in AI context isn't the same thing we think about intuitively.

--

> If it’s out of domain we know it’ll fail.

Below goes question which is out of domain. Yet LLMs handle the replies in what appearing as logical way.

``` Kookers are blight. And shmakers are sin. If peker is blight and sin who is he? ```

It is out of domain and it does not fail (I've put it through thinking gemini 2.5). Now back to article. Is observed logic intristic to LLMs or it's an elaborate form of a pattern? Acoording to article it's a pattern.