Comment by deepsquirrelnet

I'm not sure anybody but people inside the company know if the model itself is a contribution. There's no publication and no architectural details. There's no benchmarks or comparisons published. You can do all of the things they claim with an LLM, not that I think that's what they did.

Likely they have some encoder (eg ModernBERT) trained to do late interaction or latent states along the lines of ColBERT, Perceiver IO or poly-encoders.