Comment by troupo
2 days ago
> However, if you took that model and executed it in single user environment,
Again, are those environments in the room with us?
In the context of the article, is the model executed in such an environment? Do we even know anything about the environment, randomness, sampling and anything in between or have any control over it (see e.g https://news.ycombinator.com/item?id=44528930)?
It's very poor communication. They absolutely do not have to be non-deterministic.
The output of all these systems used by people not through API is non-deterministic.
I would also assume that in vast majority of cases people don't set temperature to zero even with API calls.
And even if you do set it to zero, you never know what changes to the layers and layers of wrappers and system prompts you will run into on any given day resulting in "on this day we crash for certain input, and on other days we don't": https://www.techdirt.com/2024/12/03/the-curious-case-of-chat...