← Back to context

Comment by tullie

5 hours ago

Makes sense. Implementation simplicity was part of the reason we didn't go for this. Currently the language is a transpiler, which maps SELECT and FROM into the retrieval stages, WHERE as post-filter stage, ORDER BY as score stage and REORDER BY to the final stage reranker. Because the text lookup is pushed through to the retriever we put it there. In the future we'll have more logic in the language compilation process which means that we can move things around a bit and under the hood they all get pushed to where they need to anyway.