← Back to context

Comment by m11a

1 year ago

Could use the notify to awake, and then the worker needs to lock the job row? Whichever worker gets the lock, acts on the message

6 comments

m11a

Reply
airocker  1 year ago

Thats exactly what we do but taking a lock takes 1 RTT to the database which means about 100ms. it limits the number of events receivers can handle. IF you have too many events, receivers will be just trying to take a lock most of the time.

  • jeeybee  1 year ago

    Of my head, you could attach a uuid or sequence number to the emitted event. Then based on the uuid or sequence you can let one or the other event consumer pick?

    Ex. you have two consumers, if the sequence number is odd, A picks it, if its even B picks.

    • airocker  1 year ago

      Great observation, I wrote a little wrongly. What we want ideally is guaranteed delivery to one random free worker. Uuid strategy is better than locking but this could mean that if one worker gets a longer job, all the others on this worker are delayed even if another worker is free.

    • dopp0  1 year ago

      I think this concept you mentioned would be called sharding? It's kinda required for apache Kafka and others.