Comment by HighlandSpring
3 hours ago
I've had the same thought as probably have many others - what solutions exist in this space?
To put it into words: a layer over an LLM that tracks provenance: the documents it has access to, the facts it has extracted, the mathematical expressions and queries it's run and finally the artifacts and results it's yielded complete with receipts for exactly how it arrived at it's findings and conclusions.
I assume legaltech products have to do it like this. Is the general pattern implemented somewhere on github?
I'm working on something around this that I'm hopeful I can open source in the next couple of weeks. It's just a research project, so not like a product that can be bought and shipped to prod right away, but I'm excited about it. I think a lot of people are working on this right now, at least in part, with things like memory layers and other approaches to "grounding", or with some of the more recent stuff around certification and governance. But there's sort of a split right now between:
- Groups focused strictly on model capability (i.e. benchmark scores)
- Groups really focused on governance, interpretability, control planes, etc
- A small but excitable minority of weirdos (me included) who think they're the same thing
A lot of the stuff that gets loosely clustered under neurosymbolic methods might be interesting if you're looking to dive in. The sort of pattern you're describing is something that a lot of people have taken stabs at from different directions, or at least have tangentially nipped at while working on related problems.
The name of the solution is RAG - retreival augumented generation. It is pretty easy to whip up your own over a corpus of material. I built one for myself within a week over a corpus of my favorite cook and homebrewing books. All you need is source material, postgress (pgvect) and some compute (llama.cpp with a decent chat model for chunking and labeling, an embedding and a rerank model. or some $ to use those via a router). When the corpus is indexed, you use your favorite coding agent and a system prompt that forces the model to answer only from search results.
Not sure why a solution is even needed. If I had to compile all the resources I had consulted for something like C programming, they wouldn't amount to a single shelf. I would say the same thing for research where the amount of sources is still very low. It's kinda reminded me of the big paper manuals and books before the internet. I don't think the whole library of congress was needed to write UNIX.
So AI being a source of truth and information is not a something that important. It's a generator and we should use it as such, getting it to map domains and repeating patterns. It should not be used as an encyclopedia or oracle.
>> If I had to compile all the resources I had consulted for something like C programming, they wouldn't amount to a single shelf.
Serious question - with something like a programming language where there is a sort of finite information available about the language and how to use it. I've found once you start moving into areas where this isn't the case like metaphysics where there is a lot more ambiguity, does the sourcing and information then become more valuable? Or are we going to finally be ok with ambiguity as humans?
I can see what you're saying and agree, but are there areas where sourcing does in fact become more important?
[flagged]