Comment by wyrdcurt

They are weights for matrix operations, so in principle you'd think they would be deterministic. In practice it's more complicated than that.

Not to be snarky or dismissive, I mean this genuinely: ask an LLM about it. I currently have a headache so I'm not up to explaining the technical details, but they are interesting and worth reading about.