Comment by ryandamm
2 years ago
Why not have a separate chat request to apology-check the responses?
Not my original idea, there was a link from HN where the dev did just that.
2 years ago
Why not have a separate chat request to apology-check the responses?
Not my original idea, there was a link from HN where the dev did just that.
Sounds like a great way to double your API bills, and maybe that's worth it, but it seems pretty heavy-handed to me (and equally not 100% watertight).
OpenAI's moderation API is free and just tells you if your query will be declined: https://platform.openai.com/docs/guides/moderation
Only allow one token to answer. Use logit bias to make "0" or "1" the most probable tokens. Ask it "Is this message an apology? Return 0 for no, 1 for yes." Feed it only the first 25 tokens of the message you're checking.