Comment by johnsmith1840

I did large scale tests temp 0 and there was still randomness with the same prompt inputs coming in.

I did this with several model apis.

GPU processing is not going to be the same from what I read but also the AI backend is doing a lot of fancy batching resulting in another layer of randomness.