← Back to context

Comment by fragmede

1 year ago

the boat is big enough to hold all the items

I asked it

> lion, goat, wolf riddle, but the boat is big enough to carry all of them

and it said it could do it in one step.

https://chat.openai.com/share/7b7a5462-7649-403d-a4f0-01c387...

ChatGPT-4 today (April 28th, 2024) still fails at it sometimes:

https://chat.openai.com/share/1bec923d-d727-42fe-ba9b-9f92b9...

This is ChatGPT-4 getting it wrong, months ago: https://chat.openai.com/share/caa37ad6-b7a8-451d-8f39-8a2c04...

This is ChatGPT-4 getting it right, today (April 28th, 2024): https://chat.openai.com/share/d2d9e63e-819e-4681-9f9f-8f77ea...

Ah, ok. My variation is it's a vegetarian wolf, a carnivorous goat, and a cabbage.

There's a few different hacks that will get it to work, but one of the more interesting is switching the nouns to emojis.

But almost none of the models ever get it on the first try, and every major model since GPT-4 can have the prompt tweaked to get it with the exception of Llama-3, which I just can't get to solve it with anything I've tried so far (and I'm not sure if it's because of extra strong associations to the standard form from the extra training run or if it lacks the core competencies, though I am starting to think it's the latter given how it responds as I point out errors).

I particularly like this variation because it requires remapping concepts in unintuitive ways based on broad abstractions, like having a goat potentially eat a wolf because of it being carnivorous.