← Back to context

Comment by nsonha

3 years ago

Parallel or not, the order is of importance in any queue system.

Unless you can guarantee that the processing time of each job is exactly the same, if you have multiple workers processing the same queue, you can’t order anything except the start time.

You can use locks to effectively break the queue into sub queues so that each sub queue is only being processed by 1 worker. Then you can order that sub queue.

  • job should be attempted inthe same order/priority they are enqueued, that's the meaning of the word "queue". That they take varrying amounts of time is another matter.

You have no ordering guarantees, so how can order be important? If 4 work items are scheduled on 4 independent workers, you have no guarantee which will start first or finish first.

  • The order matters in the sense that the 5th jobs should not be atempted before those 4.

    • I think the order matter at least because you want to have some FIFO approximation, otherwise some tasks can forever stuck in queue and never be picked up.

    • Then I think what you actually care about is scheduling fairness, and a strict ordering of execution of job 5 after job 4 is unimportant.