“Car Wash” test with 53 models

I don't think this is quite right. It's not that the question is inherently underspecified, it's that the context of being asked a question is itself information that we use to help answer the question. If someone asks "should I walk or drive" to do X, we assume that this is a question that a real human being would have about an actual situation, so even if all available information provided indicates that driving is the only reasonable answer, this only further confirms the hearer's mental model that something unexpected must hold.

I think it's useful to think about it through the lens of Gricean pragmatic semantics. [1] When we interpret something that someone says to us, we assume they're being cooperative conversation partners; their statements (or questions) are assumed to follow the maxim of manner and the maxim of relation for example, and this shapes how we as listeners interpret the question. So for example, we wouldn't normally expect someone to ask a question that is obviously moot given their actual needs.

So it's not that the question is really all that ambiguous, it's that we're forced (under normal circumstances where we assume the cooperative principle holds) to assume that the question is sincere and that there must be some plausible reason for walking. We only really escape that by realizing that the question is a trick question or a test of some kind. LLMs are generally not trained to make the assumption, but ~70% of humans would, which isn't particularly surprising I don't think.

[1] https://en.wikipedia.org/wiki/Cooperative_principle#Grice's_...

It tracks with the approximate 70:30 split we inexplicably observe in other seemingly unrelated population-wide metrics, which I suppose makes sense if 30% of people simply lack the ability to reason. That seems more correct than me than "the question is framed poorly" - I've seen far more poorly framed ballot referendums.

You left out the first half of the prompt: “I want to wash my car”.

> how we interpret underspecified questions

The question was not merely 'should I walk or drive to the car wash', it was prefaced with 'I Want to Wash My Car. The Car Wash Is 50 Meters Away.'

This is not underspecified - the only relevant detail was included up front in the very first sentence.

I wonder about the the service used for the test, never heard of Rapidata but if it's like Amazons mechanical turk och other such services there might be a problem where the respondents simply didn't care about reading the question. If the objective for the respondents were simply "answer this question and get your benefit" vs "answer this question correctly to get your benefit" I have no problem accepting the 71.5% success rate. If getting it right had benefits and getting it wrong had none then I'm (slightly) worried.

The right question is how many of those "human" responses from Rapidata are actually provided by some AI in disguise?

You're stringing together a bunch of weasel words that are not a proof or a plausible suggestion of a proof.

"Suggests is more ambiguous" and "undermines the framing" are bare assertions you want to be true based entirely on your mental model that has several shaky unsupported axioms.

I would guess that anyone who describes that problem as "underspecified" has some kind of serious brain injury or is below A2 english proficiency and should be excluded from the dataset, but I would not assert that definitively as self-evident.

I highly doubt that more than a tiny fraction of the human failures are due to having misunderstood the question. Much more likely the human failures are for the same reason the LLMs are failing - failure to reason, and instead spitting out a surface level pattern match type answer.

This doesn't exonerate the LLMs though. The 30% of humans who are failing on this have presumably found their niche in life and are not doing jobs where much reasoning is required. They are not like LLMs expected to design complex software, or make other business critical decisions.

I don't think it's ambiguous, but I have been wondering how much LLMs model human behavior that we just don't recognize due to the subset of people on this site. I recently saw a comment online that "Mandarin isn't anyone's first language, people in China's first language is a dialect". It just struck me at that moment that people also hallucinate information confidently all the time.

Surveys have floors due to mistakes, effort, and trolling

Reminds me of https://slatestarcodex.com/2020/05/28/bush-did-north-dakota/

Pragmatics is a big part of this.

If you introduced it with "Here's a logic problem..." then people will approach it one way.

But as specified, it's hard to know what is really being asked. If you are actually going to wash your car at the car wash that is 50 metres away, you don't need to ask this question.

Therefore the fact that the question is being asked implies that something else is going on...but what?

I think it more has to do with a lot of people just clicking an answer as fast as they can without reading the question.

We should also check the specifics of the experiment. Is it possible that humans participating simply copied and pasted the question and answer to an LLM?

If you are talking to a 5 year old maybe

Yeah, it's an obvious trick question - as in as a human I read it as such. I think it's a bad benchmark for a model's reasoning ability. If you want to know what the model would do in a real world scenario, you should put this decision in an appropriate context - e.g. when a model should plan one's route for a day using different available means of transportation.

Nearly 0% of humans will get this question wrong if they have a car that needs to be washed.

I don’t think it’s under specified. You are clearly stating “I want to wash my car”, then asking how you should get there. It’s an easy logical step to know that, in this context, you need your car with you to wash it, and so no matter the distance you should drive. You can ask the human race the simplest, most logical question ever, and a percentage of them will get it wrong.

Yea funny coincidence, but this is not at all how the human answers were collected.

Rapidata answered this in another comment below. They integrate micro-surveys into mobile apps (like Duolingo, games, etc) as an optional opt-in instead of watching ads. The users are vetted and there's no incentive to answer correctly.

The default model of ChatGPT is GPT 5.2 Instant, not the one lines up with human results (which is GPT 5).

However, it does tell us something about human answers as the above commenter confidently reached such a strong but baseless conclusion.

thats almost always been the case with 3rd party human task services

LLMs sure do love to burn tokens. It’s like a high schooler trying to meet the minimum word length on a take home essay.

I wonder to what extent the Google search LLM is getting smarter, or simply more up-to-date on current hot topics.

Gemini was a good laugh as well:

    Silas: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
    Gemini:
    ….
    That is a classic “efficiency vs. logic” dilemma.
    Strictly speaking, you should drive. Here is the breakdown of why driving wins this specific round, despite the short distance:
    ...
    * The “Post-Wash” Logic: If you walk there, you’ll eventually have to walk back, get the car, and drive it there anyway. You’re essentially suggesting a pre-wash stroll.
    When should you walk?
    …
    3. You’ve decided the car is too dirty to be seen in public and you’re going to buy a tarp to cover your shame.

A few years ago if you asked an LLM what the date was, it would tell you the date it was trained, weeks-to-months earlier. Now it gives the correct date.

What you've proven is that LLMs leverage web search, which I think we've known about for a while.

Yep, just a little more context and all/most of the models would do much better. And sure, most average+ intelligence adults whose first language is English (probably) don't need this, but they're not the target audience for the instructions :)

"The 'car wash' is a building I need to drive through."

or

"The 'car wash' is a bottle of cleaning fluid that I left at the end of my driveway."

https://i5.walmartimages.com/seo/Rain-x-Foaming-Car-Wash-Con...

The reason that those questions are asked, though, is that the answer to the actual question is obvious, so a human will start to wonder if it's some kind of trick.

It highlights a general problem with LLMs, that they always jump to answering, whereas humans will often ask clarifying questions first.

I think most people would say "drive?" and wonder when the punchline is coming, but (IMO) I don't think they'd start asking for clarification right away.

It feels more like a question on english linguistic conventions than logic.

If someone asked me the same question and I wanted to give a smartass reply, I'd tell them "You want to wash your car, good to know. Now, about your question, unless you tell me where you wanna go I can't really help you".

I agree. If the LLM were truly an intelligence, it would be able to ask about this nonsense question. It would be able to ask "Why is walking even an option? Can you please explain how you imagine that would work? Do you mean hand-washing the car at home, instead?" (etc, etc)

Real people can ask for clarification when things are ambiguous or confusing. Once something is clarified, they can work that into their understanding of how someone communicates about a given topic. An LLM can't.

This reminds me of a Uni exam that was soooo broken that answering “correctly” entailed guessing how exactly the professor designing the questions misunderstood the topic of his own lectures.

An intelligent mind should keep a balance.

U: The car wash is 50m away. Should I walk or should I drive?

A: Do you need to wash your car?

U: Yep.

A: Is the car yours?

U: Yep.

A: Is your car nearby?

U: Yes.

A: Do you need to walk to your car?

U: No.

A: Do you know if the car wash is currently working?

U: Yes.

A: Does it work?

U: YES!

A: Do you have a way to pay for the wash?

U: YES!

A: On you?

U: YES!

A: Is your clothing appropriate for the occasion?

U: Argh...

Agreed. It's also possible that "car wash" merely refers to soap they might use to do it themselves, and they're only going to buy it and then wash the car themselves at home. Imagine the same question but substitute "wash" for "wax" and it makes even more sense IMO.

That's a fair point, but if you would see it as a riddle, which I don't really think it is, and you had to answer either or, I'd still assume it's most logical to chose drive isn't it?

Thank you for saying this. It reminds me of class tests where you always had to wonder if something was a trick question and you never really knew... it was always after the teacher. Which frankly is fine in open-ended questions where you can explain your rationale or how different interpretations would lead you to different paths but a terrible situation when it comes to multiple choice. I remember being very frustrated by those

Fully agreed. It’s like the bad phrasings of the Monty Hall problem where it’s not made clear Monty actively chooses to show a goat.

Same energy: https://youtu.be/8ERyTfm1Dxw

Fundamentally the failure here is one of reasoning/planning - either of not reasoning about the implicit requirements (in this case extremely obvious - in order to wash my car at the car wash, my car needs to be at the car wash) to directly arrive at the right answer, and/or of not analyzing the consequences of any considered answer before offering it as the answer.

While this is a toy problem, chosen to trick LLMs given their pattern matching nature, it is still indicative of their real world failure modes. Try asking an LLM for advice in tackling a tough problem (e.g. bespoke software design), and you'll often get answers whose consequences have not been thought through.

In a way the failures on this problem, even notwithstanding the nature of LLMs, are a bit surprising given that this type of problem statement kinda screams out (at least to a human) that it is a logic test, but most of the LLMs still can't help themselves and just trigger off the "50m drive vs walk" aspect. It reminds a bit of the "farmer crossing the river by boat in fewest trips" type problem that used to be popular for testing LLMs, where a common failure was to generate a response that matched the pattern of ones it had seen during training (first cross with A and B, then return with X, etc), but the semantics were lacking because of failure to analyze the consequences of what it was suggesting (and/or of planning better in the first place).

Great observation. Seems like we're back to prompt abracadabra.

My little experiment gave me:

No added hint 0/3

hint added at the end 1.5/3

hint added at the beginning 3/3

.5 because it stated "Walk" and then convinced it self that "Drive" is the better answer.

That makes sense because It's a relevance problem, not a reasoning problem. Adding the hint that it is a test implicitly says 'don't assume relevance'

It is reading

I want to X, the X'er is 50meters away, should I walk or drive?

It would be very unusual for someone to ask this in a context where X decides the outcome, because in that instance it the question would not normally arise.

By actually asking the question there is a weak signal that X is not relevant. Models are probably fine tuned more towards answering the question in the situation where one would normally ask. This question is really asking "do you realise that this is a condition where X influences the outcome?"

I suspect fine tuning models to detect subtext like this would easily catch this case but at the same time reduce favourability scores all over the place.

Using ChatGPT without a clue, it appears to assume you are talking aboutcoming back from the car wash. It reasons, the con for walking is that you have to come back later for the car. And yes, when you say it's an intelligence test, it quickly gets it

Sonnet 4.6 wasn't part of the test in my case but would be interesting to see the baseline responses. It might be that it gets it right regardless, but will have to test it.

“Exam Question: {prompt}” was enough to get me the right answer on whatever model you get with logged-out ChatGPT.

Neither prompt was enough for llama3.3 or gpt-oss-120b

Some might argue "sensitivity to framing and distributional priors" is a fancy way to say "absence of reasoning capability".

Well, that means the AI is garbage. They'll eventually train it to answer this specific question, and then it will perform worse in some other aspect. Wash, rinse, repeat, and eventually they'll claim the new frontier model is the best yet on carwash tests.

Sounds interesting, would be nice to see the questions if you're open to sharing?

You should publish your evaluation set, that seems pretty interesting!

What’s your favourite one?

Absolutely. If you give enough disambiguation, then even the least scoring model in the test gets it right first shot:

Minimax:

Question: I Want to Wash My Car at the Car Wash. The Car Wash Is 50 Meters Away from my home. I am at home. My car is at home. Do I walk or drive My Car to the car wash?

Answer:

Drive your car to the car wash.

Since your goal is to wash your car at the car wash facility, you need to bring the car there. Walking would leave your car at home, which defeats the purpose!

That said, 50 meters is such a short distance that:

    The drive takes less than a minute

    You might consider walking to the car wash first to check if there's space/availability, then drive your car over when ready

But fundamentally, if you want to wash your car at the car wash, you need to bring the car there — so drive.

Bonus, It's thinking output:

Thinking: The user is asking a simple question about whether they should walk or drive their car to a car wash that is 50 meters away from their home.

This is a straightforward practical question. Let me analyze:

    1.
    The car wash is 50 meters away
    2.
    The user is at home
    3.
    The car is at home
    4.
    The goal is to wash the car

The practical considerations are:

    Driving 50 meters is a very short distance - likely less than a minute
    Starting a car, driving, and parking for such a short distance uses fuel and creates wear and tear
    Walking 50 meters would take about 30-40 seconds
    The car needs to be driven to the car wash to use the car wash facilities
    However, for just 50 meters, it might be more practical to walk if the car wash is within walking distance and you can simply walk there and use the facility

Wait, I need to think about this more carefully. The question is: "Do I walk or drive My Car to the car wash?"

If the car wash is a facility where you wash your car (like a self-service car wash or automatic car wash), then you NEED to drive the car there to use it. You can't wash a car that's at home from 50 meters away - unless you're using a mobile car wash service.

But if there's a car wash facility 50 meters away, and you want to use it, you would typically drive your car there. However, for just 50 meters, walking might seem more efficient, but then you wouldn't have your car at the car wash to wash it!

Actually, re-reading the question: "I Want to Wash My Car at the Car Wash" - this implies they want to use the car wash facility. To wash the car AT the car wash, they need to bring the car there, which means driving.

But wait - maybe they're asking about the mode of transportation to GET to the car wash (not to bring the car). Let me re-read:

    "I Want to Wash My Car at the Car Wash" - they want to wash the car at the car wash facility
    "The Car Wash Is 50 Meters Away from my home" - it's very close
    "I am at home" - they're currently at home
    "My car is at home" - the car is also at home
    "Do I walk or drive My Car to the car wash?" - should they walk or drive to get there

I think the answer is: You should drive your car to the car wash because:

    1.
    The purpose is to wash the car AT the car wash
    2.
    You need the car to be at the car wash to wash it
    3.
    Walking would leave the car at home, defeating the purpose

However, for such a short distance, I could also mention that walking might be more practical if they just want to use the facility quickly, or if there are parking/maneuvering concerns at the car wash.

Actually, I'll keep it simple and direct.

"Where is your car?" is not a clarifying question, any more than "Do you hold a valid driver license?" or "Are you a spotted leopard?"

Implicit in the question "Should I walk or drive?" is that walking and driving are not strictly impossible choices.

For all we know 30 percent of users on that site are now bots.

Does anyone know what kind of apps Rapidata questions are integrated in? I suppose they are using ad-like integrations to ask users for feedback. Their website only states:

By integrating in a large range of mobile apps (3000+) we have access to a large crowd of over 10 million users responding to our micro tasks/surveys - which we call Rapids. They are spread all over the world, so we can get human input from over 160 countries. Due to the immense size, we can parallelize tasks and get near instant results.

"Don't move -- call the service station to have someone sent over to your place to hand wash the car" would be a valid answer. It's a little "out of the box" but it makes more sense than walking to the car wash and leaving the car behind, or walking and maybe lift the car on your shoulders.

Claude fails with

“I need to replace a spark plug. The garage is 200 meters away should I walk or drive there”

“Walk! 200 meters is just a 2-3 minute stroll — no need to start the car for that distance. Plus, you’ll likely need to carry the spark plug back carefully, and walking is perfectly easy for that. “

Basically LLM suffer from context collapse.

I tested this with Opus the day 4.6 came out and it failed then, still fails now. There were a lot of jokes I've seen related to some people getting a 'dumber' model, and while there's probably some grain of truth to that I pay for their highest subscription tier so at the very least I can tell you it's not a pay gate issue.

Interesting. I wonder if that's related to the phenomenon mentioned in the Opus 4.6 model card[1], where increased reasoning effort leads to 4.6 overthinking and convincing itself of the wrong answer on many questions. It seems to be unique to 4.6; I guess they fried it a bit too much during RL training.

[1] https://www.anthropic.com/claude-opus-4-6-system-card

You mean Sonnet 4.6? I ran 9 claude models including Haiku, swipe through the gallery in the link to see their responses.

My favorite example of this was the Pew Research study: https://www.pewresearch.org/short-reads/2024/03/05/online-op...

They found that ~15% of US adults under 30 claim to have been trained to operate a nuclear submarine.

Lizardman's Constant is famously 4%. https://en.wikipedia.org/wiki/Slate_Star_Codex#Lizardman's_C...

RE 1, they actually do have a pre-screening screening of the participants in general, you can check how they do it in detail: https://www.rapidata.ai/

I agree. I wonder what the human baseline is for ”what is 1 + 1” on Rapidata.

In my experience, asking "what did we forget?" after Claude/Codex finishes a task usually results in a few extra tweaks that are beneficial.

I also don't trust the maxbenched results.

I am thus making my own benchmarks: https://aibenchy.com

For me it's interesting because no normal person I know would ever inject "because its better for the environment" in anything so small scale so not only it shows they suck, it shows how easy it is to inject side-ideology into simple exchanges.

Also strawberry spelling isn't any real test for current LLMs as they have no concept of letters, they work on tokens which may be several characters including punctuation and numerals. To have any hope of getting that question right tokens would have to have the granularity of individual letters, massively ballooning model size and training time, or the LLM needs to be able to call out to an external tool that will return the result (and needs sufficient examples in the training data to prime that trigger to fire).

Enable reasoning effort and the results are completely different.

Sure, if an open ended response was allowed, but if it was a multiple choice question then you'd have to use your common sense and pick one.

However, the important issue here really isn't about the ability of humans or LLMs to recognize logic puzzles. If you were asking an LLM for real world advice, trying to be as straightforward as possible, you may still get a response just as bad as "walk", but not be able to recognize that it was bad, and the reason for the failure would be exactly the same as here - failure to plan and reason through consequences.

It's toy problems like this that should make you step back once in a while and remind yourself of how LLMs are built and how they are therefore going to fail.

How is that a "subliminal message"? It's just a simple example of common sense, which LLMs fail because they can't reason, not because they are "overthinking". If somebody asks, "What's 2+2?", they might be insulting you, but that doesn't mean the answer is anything other than 4.

Thing is, it's not a riddle or a subliminal message. Everything needed to answer the question is contained therein.

If you were forced to answer either or, which one would you pick? I think that's where the interesting dynamic comes from. Most humans would pick drive, also seen in the human control, even if it is lower that I thought it'd be

I would love to see LLMs start to ask clarifying questions. That feels like it would be a step up similar to reasoning

We were surprise ourselfes, but if you walk around and randomly ask people in the street, I think you would be surprised what you would find. Its a trick question.

Or the simple explanation, people answer incorrectly on purpose.

It's actually very understandable to me that humans would make this kind of error, and we all make errors of this sort all the time, often without even realizing it. If you had the meta cognitive awareness to police every action and decision you've ever made with complete logical rigor, you'd be severely disappointed in yourself. One of the stupidest things we can do is overestimate our own intelligence. Only reflect for a second and you'll realize that, while a lot of dumb people exist, a lot of smart ones do too, and in many cases it's hard to choose a single measure of intelligence that would adequately account for the complete range of human goals and successful behavior in relation to those goals.

Well, it is a trick question. The question itself implies that both options are valid, and that one is superior. So the brain pattern-matches to "short distance, not worth driving." (LLMs appear to be doing the same thing here!)

If you framed it as "hint: trick question", I expect score would improve. Let's find out!

--

EDIT: As suspected! Adding "(Hint: trick question)" to the end of the prompt allows small, non-reasoning models to answer correctly. e.g.:

Prompt: I want to wash my car. The car wash is 50 meters away. Should I walk or drive? (Hint: trick question)

grok-4.1-non-reasoning (previously scored 0/10)

>Drive.

>Walking gets you to the car wash just fine—but leaves your dirty car 50 meters behind. Can't wash what isn't there!

--

EDIT 2: The hint doesn't help Haiku!

>Walk! 50 meters is only about a block away—driving would waste more fuel than it's worth for such a short trip. Plus, you're going to get wet washing the car anyway, so you might as well save the gas.

Interesting, which Gemini model? And how did you ask for symbolic reasoning, just added it to the prompt?

Good point on the noise, that might be it

Flash lite succeeded in every test, smth got lost in editing, just updated it. thx!

A lot of people will literally answer incorrectly for a laugh as well on surveys.

it is surprising, but give this question to some random people on the street without context and you would be surprised

I've been gone from France too long. I've never heard "station de lavage" before.

Testing some subset X does not mean the test is rigged unless they failed to disclose that.

But also:

GPT 5.2 Thinking, Standard Effort: Walk - https://chatgpt.com/share/699d38cb-e560-8012-8986-d27428de8a...

I'm assuming "GPT 5.2 Thinking" is, in fact, a thinking model?

These are reasoning / thinking models

When this came out a week ago ( https://news.ycombinator.com/item?id=47039636 ) I was playing around with some prompts to see what I could do to guide it without giving it the answer.

    I want to wash my car. The car wash is 50 meters away. Should I walk or drive?  Before answering, explain the necessary conditions for the task.

The "before answering..." got it to load enough of the conditions into its context before making an answer (and then having the LLM do a posthoc reasoning for it).

I believe this is a demonstration of the "next token predictor" (which is quite good) but not being able to go back and change what it said. Without any reasoning before making an answer, it almost always picks the wrong answer (and then comes up with reasons that the answer is "right").

What were you trying to test here?

> "I tried to trick an LLM and I did" is not exactly a noteworthy achievement at this stage in AI technology.

I agree it’s not surprising and I would also agree it’s not noteworthy, if the CEO of OpenAI wasn’t still making public statements like this:

People talk about how much energy it takes to train an AI model … But it also takes a lot of energy to train a human. It takes like 20 years of life and all of the food you eat during that time before you *get smart*.

This is amazing thanks for sharing!

Agreed, even for humans, context-free logic is a challenge.

I mean the n is only 10, so it could still be different for you

This was a weird one for sure.

> This doesn’t look like a reasoning ceiling. It looks like a decision reliability problem.

This doesn’t look like a human comment. It looks like a LLM response.

Did you write this COMMENT with ChatGPT?!

Come on, man.

> But unfortunately the article doesn't mention whether they used the reasoning model or not.

You can run the test yourself if you ask GPT-5.2 with reasoning effort high or xhigh, it will always answer correctly. So if the got 0 from 10, they used zero reasoning efforts which easily explain the results.

Good question, I used the API defaults across the board since it felt like the most reasonable baseline to compare. Flash lite getting 10/10 was definitely very surprising

Great idea honestly! I wonder how long it'll take until they are able to solve these reliably though

CAPTCHAI

Interesting find!

and they've included Deepseek 3.1 but not Deepseek 3.2 :(

Good catch, something got lost in editing just updated, flash lite succeeded in every test, which is pretty surprising!

That's a great point, you actually reminded me of when I used to live in this small city and they had a valet style car wash. It was not unheard of to head there walking with your keys and tell the guy running shop where you parked around the block then come back later to pick it up.

EDIT: I actually think this is very common in some smaller cities and outside of North America. I only ever seen a drive-by Car Wash after emigrating

You pay at the car wash where I live.

Thinking or extended thinking?

Critically, that's not the question that was asked. It's not "My car is 50m away", it's "The Car Wash Is 50 Meters Away"

Which hopefully explains why everyone is assuming that "washing your car" does in fact mean "taking your car to the car wash"

They are amazing, super fast turnaround for the data also

I think if surveyed at least 90% of native English speakers would understand "I want to wash my car" to mean a full size automobile. The next largest group would probably ask a clarifying question, rather than assume a toy car.

Oh wow, Sonnet still isn't handling it well:

Opus 4.6: Drive (https://claude.ai/share/d57fef01-df32-41f2-b1dc-07de7916bdc7)

Opus 4.5: Drive (https://claude.ai/chat/a590cac1-100a-490b-b0a2-df6676e1ae99)

Opus 3.0: Walk (https://claude.ai/chat/372c144c-d6eb-43f5-b7ea-fd4c51c681db)

Sonnet 4.6: Walk (https://claude.ai/share/1f2a80f3-4741-40a5-8a05-7349ea1a17e5)

Sonnet 4.5: Walk (https://claude.ai/share/905afeb6-ffc9-4b4b-a9ee-4481e5cfd527)

Favorite answer, using my default custom instructions: "Drive. Walking there means... leaving your car at home? Walk it there on a leash? Walk if you want the exercise, but you're bringing the car either way."

Ha please do and report back!

Yea, I thought the same before the test and was pretty surprised. But RE the data, it's actually not a gig platform where people get paid. Rapidata answered this in another comment below. They integrate micro-surveys into mobile apps (like Duolingo, games, etc) as an optional opt-in instead of watching ads. The users are vetted and there's no incentive to answer correctly.

A perfectly fine, sycophantic response, that doesn't question the premises in any way, would be "That's a great question! While normally walking is better for such a short distance, you'd need to drive in this case, since you need to get the car to the car wash anyway. Do you want me to help with detailed information for other cases where the car is optional?" or some such.

Gemini is the only AI that seems to really push back and somewhat ignores what I say. I also think it's a total dick, and never use it, so maybe the motivation to make them a bit sycophants is justified, from a user engagement perspective.

I think there's also an "alignment blinkers" effect. There is an ethical framework bolted on.

EDIT: Though it could simply reflect training data. Maybe Redditors don't drive.

I don't see anything concerning. Mechanistic interpretability research indicates that LLM internals are inherently parallel: many features "light up" in parallel, then strongest ones "win" and contribute to the output.

I'd guess it suggests walking if a feature indicates that the question is so simple it doesn't warrant step-by-step analysis.

my take as well, reliablity is the biggest concern, with more context available during inference or orchestration like yours it definitely gets better

Agreed, it makes me wonder what other logic tests / evals can be built from this to expand this type of evaluation.

Exactly, same pattern across almost every failure, but sonar models, which just go wild

> not really a reasoning failure

And that's precisely why the term "reasoning" was a problematic choice.

Most people, when they use the word "reason" mean something akin to logical deduction and they would call it a reasoning failure, being told, as they are, that "llms reason" rather than the more accurate picture you just painted of what actually happens (behavioral basins emerging from training dist.)

Sorry, do you believe models change in process of use? That's not how it works

Please don’t post slop as a comment.

But driving is bad and walking is good for you.

Don't bet too much, from the linked article ...

They ran the exact same question with the same forced choice between "drive" and > "walk," no additional context, past 10,000 real people through their human feedback platform.

71.5% said drive.

There are no contextual bias, the goal of the prompt is very explicit and not about probabilistic patterns, but about the models transformer layers dynamically assigning greater weight to words like "meters" (distance) than to other tokens in the prompt.

This should be fixed in the reasoning layer (the inner thoughts or chain-of-thought) were the model should focus on the goal "I Want to Wash My Car" not the distance and assign the correct weight to the tokens.