Comment by est
1 year ago
The most simple job queue in MySQL:
update job_table set key=value where ... limit 1
It's simple and atomic. Unfortunately PG doesn't allow `update ... limit` syntax
1 year ago
The most simple job queue in MySQL:
update job_table set key=value where ... limit 1
It's simple and atomic. Unfortunately PG doesn't allow `update ... limit` syntax
> It's simple and atomic
and almost certainly incorrect, gotta read at least https://www.pgcon.org/2016/schedule/attachments/414_queues-p... which discusses FOR UPDATE SKIP LOCKED
that's a nice read, but does it also apply to MySQL (InnoDB)?
I've done this with mysql. Never do one at a time if you have jobs per minute over 30. It won't scale. Instead have the job dispatcher reserve 100 at a time and then fire that off to a subprocess which will subsequently fire off a process for each job. A three layer approach makes it much easier to build out multiserver. Or if you don't want the headache just use SQS which is pretty much free under 1 million jobs.
3 replies →