Comment by bayindirh

> Enabling pre-emption in your queues by default and that'll change.

We run preemptive queues, and no. Not all jobs are compatible with that. Esp. the code researchers developed themselves.

My own code also doesn't have support for checkpointing. Currently it's blazing fast, but for bigger jobs it might need the support, and it needs way more cogs inside the pipeline to make it possible.