Comment by Scaevolus
10 months ago
This is an extremely bad method of storing text data. Video codecs are not particularly efficient at compressing QR codes, given the high contrast between the blocks defeating the traditional DCT psychovisual assumptions of smooth gradients. There is little to no redundancy between QR code encodings of similar text.
You'd probably have a smaller database and better results crunching text into a zip file, or compressed rows in a sqlite database, or any other simple random-access format.
I'd say it be bewildering if there were not a more efficient way to store text for the purpose in context, than "QR codes in compressed video frames".
The vector database previously used must have been very inefficient.
> The vector database previously used must have been very inefficient.
Especially if it was taking ~800 ms to do a search. At that speed, you'd probably be better off storing the documents as plain text, without the whole inefficient QR/H264 round-trip.