Comment by HanClinto
4 days ago
Looking at llama.cpp, it already supports the logprob field in its OAI API emulation, so it shouldn't be too difficult to use this library with it.
It feels like this would be useful enough to build around -- I especially like the idea of asking the API to return the top K results for each field, and denoting their likelyhood -- almost like a dropdown box with percentages attached for each possible result.
I believe mikupad[0] supports showing logprobs from a llama.cpp backend.
[0]:https://github.com/lmg-anon/mikupad
Thank you for this link -- I had not seen this before. That is an absolutely gorgeous and intuitive interface!