Comment by the8472
18 hours ago
shipping base64 in json instead of a multipart POST is very bad for stream-processing. In theory one could stream-process JSON and base64... but only the json keys prior would be available at the point where you need to make decisions about what to do with the data.
Still, at least it's an option to put base64 inline inside the JSON. With binary, this is not an option and must send it separately in all cases, even small binary...
You can still stream the base64 separately and reference it inside the JSON somehow like an attachment. The base64 string is much more versatile.
> Still, at least it's an option to put base64 inline inside the JSON. With binary, this is not an option and must send it separately in all cases, even small binary...
There's nothing special about "text" or binary here. You can absolutely put binary inside other binary; you use a symbol that doesn't appear inside the binary, much like you do for text.
You use a divider, like " is for json, and a prearranged way to avoid that symbol from appearing inside the inner binary (the same approach that works for text works here).
What do you think a zip file is? They're not storing compressed binary data as text, I can tell you that.
Binary usually means arbitrary byte sequences so you can't choose a single delimiting character. The usual approaches are storing the length somewhere or picking a sufficiently long random sequence that it's vanishingly unlikely to occur in the payload.
This reminds me that I just learned the other day that .a files are unix archives, which have a textual representation (and if all the bundled files are textual, there's no binary information in the bundle). I thought .a was just for static libraries for the longest time, and had no idea that it was actually an old archive format.
1 reply →
Even with binary, you can store a binary inline inside of another one if it is a structured format with a "raw binary data" type, such as DER. (In my opinion, DER is better in other ways too, and (with my nonstandard key/value list type added) it is a superset of the data model of JSON.)
Using base64 means that you must encode and decode it, but binary data directly means that is unnecessary. (This is true whether or not it is compressed (and/or encrypted); if it is compressed then you must decompress it, but that is independent of whether or not you must decode base64.)
I don't get why using a binary protocol doesn't allow handling strings. What's the limitation ?