Comment by nautilus50
4 days ago
+1, llama.cpp has a --grammar parameter which you can pass a BNF style grammar file to constrain generation. It can be used in Python llama.cpp wrapper
https://til.simonwillison.net/llms/llama-cpp-python-grammars
4 days ago
+1, llama.cpp has a --grammar parameter which you can pass a BNF style grammar file to constrain generation. It can be used in Python llama.cpp wrapper
https://til.simonwillison.net/llms/llama-cpp-python-grammars
Yes. But even then, the probabilities are not calibrated. In jev/laya, they are (well, relatively anyways).