← Back to context

Comment by soulofmischief

17 days ago

[citation needed]

You can set temperature to 0 in many LLMs and get deterministic results (on the same hardware, given floating-point shenanigans). You can provide a well-defined spec and test suite. You can constrain and control the output.

14 comments

soulofmischief

Reply
soraminazuki  17 days ago

LLMs produce deterministic results? Now, that's a big [citation needed]. Where can I find the specs?

Edit: This is assuming by "deterministic," you mean the same thing I said about programming language implementations being "controllable, reproducible, and well-defined." If you mean it produces random but same results for the same inputs, then you haven't made any meaningful points.

  • soulofmischief  17 days ago

    I'd recommend learning how transformers work, and the concept of temperature. I don't think I need to cite information that is broadly and readily available, but here:

    https://medium.com/google-cloud/is-a-zero-temperature-determ...

    I also qualified the requirement of needing the same hardware, due to FP shenanigans. I could further clarify that you need the same stack (pytorch, tensorflow, etc)

    • soraminazuki  17 days ago

      This gcc script that I created below is just as "deterministic" as an LLM. It produces the same result every time. Doesn't make it useful though.

          echo '#!/usr/bin/env bash' > gcc
    echo 'cat <<EOF' >> gcc
    openssl rand -base64 100 >> gcc
    echo 'EOF' >> gcc
    chmod +x gcc

      Also, how transformers work is not a spec of the LLM that anyone can use to learn how LLM produces code. It's no gcc source code.

      11 replies →