I don’t see how this follows. If you have one single-threaded server doing the job of 20 similarly specified servers running the distributed system, you could run every job twice, on two completely independent servers on two completely independent networks, and still be 10x as efficient unless both failed catastrophically during the same job. Or you could run three completely independent copies of the same job and still be at nearly 7x efficiency, etc. There is no need for any sort of “consistency mode” here. This is just brute force, without any synchronisation between servers or resumption of aborted jobs at all.
Weren’t we comparing parallelised and single-threaded data processing jobs, though? If you’re introducing external communications in real time as a factor, wouldn’t that be a fundamentally different situation?
I don’t see how this follows. If you have one single-threaded server doing the job of 20 similarly specified servers running the distributed system, you could run every job twice, on two completely independent servers on two completely independent networks, and still be 10x as efficient unless both failed catastrophically during the same job. Or you could run three completely independent copies of the same job and still be at nearly 7x efficiency, etc. There is no need for any sort of “consistency mode” here. This is just brute force, without any synchronisation between servers or resumption of aborted jobs at all.
That only works if the servers don't rely on external dependencies or if every single opertaion is idempotent.
Weren’t we comparing parallelised and single-threaded data processing jobs, though? If you’re introducing external communications in real time as a factor, wouldn’t that be a fundamentally different situation?
1 reply →