← Back to context

Comment by demosthanos

20 hours ago

The archivists themselves say that they run into such texts often enough that this program was needed:

> The agency uses artificial intelligence and a technology known as optical character recognition to extract text from historical documents. But these methods don’t always work, and they aren’t always accurate.

They are absolutely aware of the advances in these tools, so if they say they're not completely there yet I believe them. One likely reason is that the models probably have less 1800s-era cursive in their training set than they do modern cursive.

It's likely that with more human-tagged data they could improve on the state of the art for OCR, but it's pretty arrogant to doubt the agency in charge of this sort of thing when they say the tech isn't there yet.

Can someone please post a sample of one of these images that can only be read by a human for us naive OCR believers to see?

  • I've posted these above, but I'll give you your own copy because the bits are free. Does your OCR work on these? Mine sadly doesn't. But if yours does, then I'll switch to it.

    https://imgur.com/a/CDU6Lgs

    • The problem statement was text that random humans can read and OCR cannot.

      If you want to provide a good faith answer at least make it English. I assume this is French but it’s obviously much harder to evaluate on both ends when you’re mixing up the language.

  • To be fair there was a similar discussion a few days ago in which an SME remained unconvinced: https://news.ycombinator.com/item?id=42566391

    I don't necessarily agree with her conclusion because she wasn't participating directly in the thread and wasn't completely responsive to some of the points raised, but still, it appears that there are a few instances of difficult-to-read handwriting where OCR is still coming in second to skilled human interpretation.

Then please provide a single example that we can’t instantly solve. Happy to prove them wrong.