Comment by culi
2 months ago
Someone posted a project here about a month ago where they compare models in head-to-head matchups similar to llmarena
https://www.ocrarena.ai/leaderboard
Hasn't been updated for Mistral but so far gemeni seems to top the leaderboard.
OCR developers from decades past must be slapping their foreheads now that it seems users will wait a whole minute per page and be happy.
What they are happy about is accurate OCR.
Getting the wrong answer really quickly is not the best goal.
You can also sort by latency. dots.ocr has the lowest at 3.8s/page. And although it doesn't fare very well against much larger slower models, it's still streets ahead of traditional OCR techniques
How can something have a very high ELO but a very low win rate?
You don't loose any elo if your opponent is much stronger than you. Remis could in theory play a part as well.
very nice comparison! I'd like to see on what examples OCR engines fail