Comment by rightbyte

2 years ago

Note, I haven't vetted the code. Use sandbox. Something like this should get you started:

    wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf

    git clone https://github.com/ggerganov/llama.cpp

    cd llama.cpp
    make

    ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -i -ins

0 comments

rightbyte

Reply