Comment by apparent
5 days ago
What is an efficient process for doing this? For each output from LLM1, you paste it into LLM2 and say "does this sound right?"?
If it's that simple, is there a third system that can coordinate these two (and let you choose which two/three/n you want to use?
Markdown files are everything. I use LLMs to create .md files to create and refine other .md files and then somewhere down the road I let another LLM write the code. It can also do fancy mermaid diagrams.
Have it create a .md and then run another one to check that .md for hallucinations.
You can use existing guardrails software to implement this efficiently:
NVIDIA NeMo offers a nice bundle of tools for this, among others an interface to Cleanlabs API to check for thruthfullness in RAG apps.