Comment by jkarneges
4 days ago
I've had no trouble hitting the Firebase API at the speed items are created, with a 5 second delay between retries.
For scraping HN directly, in my experience you have to go extremely slow, like 1 minute between fetching items. And if you get blocked, it may be better to wait a long time (minutes) before trying again rather than exponential backoff, in order to get out of the penalty box. You'll need a cache for sure.
No comments yet
Contribute on Hacker News ↗