Comment by est

2 days ago

I hope there's some kind of memory-hungry checker to replace the CPU cost.

a 2GB memory consumption wont stop them, but it will limit the parallelism of crawlers.