Comment by EvanAnderson
16 days ago
I did a project to translate data framed in the ASCII field/record separator characters and it was gloriously easy. All the ugly escaping considerations with comma-delimited data went away and it became much easier.
What happens when the data contains the record or field separator characters?
I suppose you could document that it's unsupported, and just drop or reject such values, but then the system couldn't be used to handle test data for such systems, for example.
In the case of this system (a quasi-EDI interface used to move records from a fleet fueling point-of-sale system to the ERP software) those characters were forbidden by the source application. My code would have exploded in a fireball if they had been present, but the specification said they couldn't be.
If it's purely binary data, then you can't.
Otherwise you need to have some sort of escape mechanism, exactly like quoting strings in CSV. In fact, there's an ASCII code "ESC" for entirely that purpose. :)
The problem is that those characters are non-printable, which means if you're just dumping the file out somewhere, you can't see them.
Easy - don't
Same as any separator. Either it's not in acceptable set of non-separator input or there's an escape (that can also escape itself for the literal).