Comment by anonym29
21 hours ago
Is including a json schema validator and running the output through the validator against the input prompt, such that you can detect when the output doesn't match the schema, and optionally retry until it does match (possibly with a max number of attempts before it throws an error) too complex of an idea for the target implementation concept you were envisioning?
It certainly doesn't intuitively sound like it matches the "Do one thing" part of the Unix philosophy, but it does seem to match the "and do it well" part.
That said, I can totally understand a counterargument which proposes that schema validation and processing logic should be something else that someone desiring reliability pipes the output into.
I'm not sure. I think I need to use it more to see what the LLMs do with bad data. The design you're suggesting might be the answer though.