← Back to context

Comment by soraminazuki

17 days ago

LLMs produce deterministic results? Now, that's a big [citation needed]. Where can I find the specs?

Edit: This is assuming by "deterministic," you mean the same thing I said about programming language implementations being "controllable, reproducible, and well-defined." If you mean it produces random but same results for the same inputs, then you haven't made any meaningful points.

13 comments

soraminazuki

Reply
soulofmischief  17 days ago

I'd recommend learning how transformers work, and the concept of temperature. I don't think I need to cite information that is broadly and readily available, but here:

https://medium.com/google-cloud/is-a-zero-temperature-determ...

I also qualified the requirement of needing the same hardware, due to FP shenanigans. I could further clarify that you need the same stack (pytorch, tensorflow, etc)

  • soraminazuki  17 days ago

    This gcc script that I created below is just as "deterministic" as an LLM. It produces the same result every time. Doesn't make it useful though.

        echo '#!/usr/bin/env bash' > gcc
    echo 'cat <<EOF' >> gcc
    openssl rand -base64 100 >> gcc
    echo 'EOF' >> gcc
    chmod +x gcc

    Also, how transformers work is not a spec of the LLM that anyone can use to learn how LLM produces code. It's no gcc source code.

    • soulofmischief  17 days ago

      You claimed they weren't deterministic, I have shown that they can be. I'm not sure what your point is.

      And it is incorrect to base your analysis of future transformer performance on current transformer performance. There is a lot of ongoing research in this area and we have seen continual progress.

      10 replies →