I spent quite a lot of time on that one, for obvious reasons. But in general it is not too hard. The GCM is a block-based cypher with built-in checksum, unlike CTR, which is a streaming one. So all you need to do is have a fixed block size where you store the header and the data. The nonce is 12 bytes, gcm tag is 16 bytes, so that is fixed 28 bytes. After some experimenting, 64kb block size seemed to work the best, despite it being quite a large chunk of data. And then, as you know, you have exactly 64kb of data in each chunk, you just stack them one after another. The hard part is then handling reads as you need to know into which chunk you have to seek, decrypt it and then seek to the correct position to stream/read the correct data. And once you reach the end of the chunk to move on to the next one. It is a bit tricky but perfectly doable and have been working for me for probably 3 years now. One caveat is to properly handle the last chunk as that one will not be full 64kb but whatever was left in the buffer of the last data. This is important for appending to existing files.
I've been just re-encrypting to CTR and streaming from that. You can stream ok from a big, single GCM file, but random-access has to faked by always restarting at 0...
Problem with CTR is that it is not a block-based cypher, which means you cannot append to existing file. For example if you have multipart file uploads, this would just not work. Also CTR lacks checksum integrity, it only XORs the bytes.
And yeah, like I said, random access is possible but you have to write your own "driver" for it.
I spent quite a lot of time on that one, for obvious reasons. But in general it is not too hard. The GCM is a block-based cypher with built-in checksum, unlike CTR, which is a streaming one. So all you need to do is have a fixed block size where you store the header and the data. The nonce is 12 bytes, gcm tag is 16 bytes, so that is fixed 28 bytes. After some experimenting, 64kb block size seemed to work the best, despite it being quite a large chunk of data. And then, as you know, you have exactly 64kb of data in each chunk, you just stack them one after another. The hard part is then handling reads as you need to know into which chunk you have to seek, decrypt it and then seek to the correct position to stream/read the correct data. And once you reach the end of the chunk to move on to the next one. It is a bit tricky but perfectly doable and have been working for me for probably 3 years now. One caveat is to properly handle the last chunk as that one will not be full 64kb but whatever was left in the buffer of the last data. This is important for appending to existing files.
I've been just re-encrypting to CTR and streaming from that. You can stream ok from a big, single GCM file, but random-access has to faked by always restarting at 0...
Problem with CTR is that it is not a block-based cypher, which means you cannot append to existing file. For example if you have multipart file uploads, this would just not work. Also CTR lacks checksum integrity, it only XORs the bytes.
And yeah, like I said, random access is possible but you have to write your own "driver" for it.