Comment by locusofself
16 hours ago
wow, they had to OCR it back in from paper printouts
> This source code is old enough that it hadn’t been stored digitally. “A dedicated team of historians and preservationists led by Yufeng Gao and Rich Cini,” calling itself the “DOS Disassembly Group,” painstakingly transcribed and scanned in code from paper printouts provided by Paterson. This process was made even more difficult because modern OCR software struggled with the quality of the decades-old printout.
I'd like to hear more about what works in OCR of dot-matrix fonts.
I've been able to OCR letter-quality printer output to 97% (mostly Os and Xs problems).
But it seems that machine-learning text-recognition is also now biased to reject computer code because it doesn't look like human language.
There's a writeup here from one of the people on the team about the work it took to go from the listings to source code. http://cini.classiccmp.org/recoveryblog.htm
> With less-than-satisfactory OCR output, I resorted to a process I used many years ago when converting scans made of old Commodore ROM dumps printed on a Commodore 1515 dot-matrix printer. The process relies on the ASCII OCR output having the same repetitive errors. "B" and "8", "S" and "5" are good examples, as are "l" and "1", and "O" and "0". There are many other similar single-character errors and, when working with x86 code, there are similar errors with instructions like "MOV". This process naturally works better if the output file is monolithic rather than single-page OCR conversions because you can do substitutions across the entire converted printout and not 75 separate files.
> The next formatting hassle was the spacing. This required repetitive substitutions of a descending numbers of spaces to tabs (i.e., replace 8 spaces with a tab, 7, 6, etc.). Then if you want to return it to fixed spaces (which is likely how the original printer printed it -- spaces and not vertical tabs), you can. For pure re-creation work, spaces produce absolute column formatting while tabs can move around depending on the program displaying the file.
> Once you run thought the 15 or so common global substitutions and tab conversion, it's a lot easier to work with the file to fix formatting and perform other cleanup. This is then followed by a line-by-line comparison against the original printouts. Overall I'd say the conversion output quality with this method is very good.
Hmm, doesn't say anything about what OCR tools they used.
I've got a 4" stack of wide-carriage COBOL. I guess it's two revisions of the same system so I only need to scan the newer half. Its probably from a TI Omni 810.
On the other hand, I've got 100 pages of code printed in compressed font by someone wanting to make sure that 80+ char lines fit within margins. So a lot of words just don't come out at all. A frequent error is "A" becomes "H", "O" becomes "U" because the top dots aren't "attached".
And columns of line numbers starting with 0001, or hex? The most confounding thing is OCR that thinks 00 is a sideways 8, and that dominates the uniform block, so it tries to interpret the whole column as sideways text. In another situation, it interprets two stacked lines (each starting with 0) as one line starting with 8 and it just goes off the rails.
So I've been working with automatic skew correction, then clipping it into rows, in order to get each line of text isolated from the surrounding context. When I do that, I get better results, but it is not great either.
I'm considering going all-in on training a new recognizer on snippets. For that, I'll be constructing "The Set of All As" and so on.
Pretty interesting. I wonder if a whitelist against certain columns in the output could help, e.g. this column can only contain valid x86 instructions (e.g. MOV is allowed, M0V is not), this column can only contain hexadecimal (1 is allowed but never "l"), etc. Probably more work than it's worth given the final line-by-line comparison that happens anyway.
Boring reply perhaps, but I've had wild success with adding even a tiny LLM afterwards to do "fixups" over OCRd text, works great for the typical O/0 issues and similar, just pass it the scrambled OCRd text together with the text around it, and even dumb and tiny 7b models running on CPU do a pretty fine job.
ABBYY has a specific module for dot matrix printouts so I’m surprised it was a struggle for them but every document is different
I've recovered some ancient software I wrote via scanning in listings I found among my dad's papers.
Yet another case where text printed on paper outlived any digital storage.
Seems like it was never digitally stored in the first place, and the printed text was barely readable due to age. Not really a big win for paper.
Well it had to have been on disk or tape at some point. It wasn't all typed in by hand every time they needed to build a new version.
15 replies →
The idea that it never existed digitally is obviously untrue. Likely poor wording in the author's part. They probably meant something like, so old that a printout is all that survived (which sounds vaguely like not being digital to someone in an era so far removed from a time when programs were/could realistically be printed.)
4 replies →
Early versions of some things, MS Basic being one example I think, were baked into ROM. One of the best innovations that Paul Allen came up with was adding software hooks to the code so bugs that were found later could still be patched.
How did they print it then, I wonder?
3 replies →
One has to be pretty ignorant and dismissive to claim that this is not "a big win for paper".
First of all, that comment is weirdly out of place. The quality and longevity of paper is not the topic.
Secondly, there are fragments of paper with writing as old as 2,000 years.
Thirdly, paper you look at and see the writing. With digital documents, you need the technology to read the medium and then you need to know how the information was encoded onto the medium, before you even arrive at the same level with paper, where you can start to decide the actual writing.
Paper has brought us where we are today, and given us what we know about the past. Don't be so ignorant and dismissive.
> struggled with the quality of the decades-old printout.
barely
It sounds like this printout has deteriorated badly and was barely readable.
If it was your standard issue cheap dot-matrix printout, it may not been particularly legible even back then.
1 reply →