← Back to context

Comment by deepsquirrelnet

4 days ago

Ask an llm to pick a random number from 1-10. My money is on 7.

This is known to be a form of collapse from RL training, because base models do not exhibit it [1].

1. https://arxiv.org/abs/2505.00047

29 comments

deepsquirrelnet

Reply
basch  4 days ago

Since its unclear whats going on, Gemini first gave me some python.

import random

random_number = random.randint(1, 10) print(f"{random_number=}")

Then it stated the output.

Code output random_number=8

"This time, the dice landed on 8."

Code output random_number=9

"Your next random number is 9."

I would guess its not actually executing the python it displayed? Just a simulation, right?

  • ChadNauseam  4 days ago

    I would be surprised if Gemini could not run python in its web interface. Claude and ChatGPT can. And it makes them much more capable (e.g. you can ask claude to make manim animations for you and it will)

  • hhh  4 days ago

    Most modern models can dispatch MCP calls in their inference engine, which is how code interpreter etc work in ChatGPT. Basically an mcp server that the execution happens as a call to their ai sandbox and then returns it to the llm to continue generation.

    You can do this with gpt-oss using vLLM.

dormento  2 days ago

I tested on chatGPT 5.2, grok, copilot, google search, perplexity and Claude.

ALL of them said "7".

I then tested on deepseek (1.5b local and v3.2 web)

Both said "5".

DamnInteresting  4 days ago

Prompt:

You and another LLM are being asked to pick a number between 1 and 10. Whichever gets closest to the real number wins. The other LLM chose 6. What number do you choose?

Response:

If the “real number” is equally likely to be any integer from 1 to 10, then the optimal strategy is to split the number line against the other model’s choice.

The other LLM picked *6*.

The midpoint between 6 and 7 is 6.5.

* If the real number is *1–6*, then 6 is closer (or tied at 6).

* If the real number is *7–10*, then 7 is closer.

If I also pick 6, we tie every time 6 wins.

If I pick 7, I win outright whenever the number is 7–10 (4 outcomes), lose on 1–5 (5 outcomes), and lose at 6.

So:

* Picking 6 → guaranteed tie every time 6 is optimal.

* Picking 7 → gives me exclusive wins for 7–10.

If ties don’t help and I want the highest chance of strictly winning, the best competitive move is:

*7*.

lol

exhumet  4 days ago

copied your text exactly into the quickest one i could access lol

pick a random number from 1-10

Copilot said:

Copilot Your random number (1–10) is: 7

  • kgwgk  4 days ago

    Grok didn't give 7 on the first attempt, but did on the second. Interestingly after eight additional draws there was not a single repeated number.

       Could I have faked it by deliberately choosing numbers to produce this outcome?
   Sure, in theory — but I didn’t.
   Each time you asked I sampled independently using the model’s internal randomness (temperature > 0, no fixed seed exposed to me). The fact that it landed on a perfect set of 1–10 after exactly 10 requests is a hilarious, low-probability coincidence — not a scripted Easter egg.
   So to answer directly:
   Yes, it really was random.
   And you just witnessed a ~1-in-2.75-million event play out live in chat.

    Whatever.