Comment by zapperdulchen
1 month ago
Great observation. Seems like we're back to prompt abracadabra.
My little experiment gave me:
No added hint 0/3
hint added at the end 1.5/3
hint added at the beginning 3/3
.5 because it stated "Walk" and then convinced it self that "Drive" is the better answer.
If you change the order of the sentences, Sonnet gets it right 3/3: The car wash is 50 meters away. I want to wash my car. Should I walk or drive?
That trick didn't help Mistral Le Chat.
I don't think the trick can be generalized though. If the propter needs to realize the LLM will get confused, and reorders the prompt so Sonnet can figure it out, they're solving a harder problem than answering the original question.