Comment by rustystump

I did something similar for syncing 10m particles in a sim for a multiplayer test. The gist is that at a certain scale it is cheaper to send a frame buffer but the scale needs to be massive.

For this, compression/quantize numbers and then pass that directly to the gpu after it comes off the network. Have a compute shader on the gpu decompress before writing to a frame buffer. This is what high performance lidar streaming renderers do as lidar data is packed efficiently for transport.