Comment by Philpax

Yes and no - yes, in theory, but in practice, non-determinism can be introduced at different points along the stack. See Thinking Machines' post on LLM non-determinism: https://thinkingmachines.ai/blog/defeating-nondeterminism-in...