Comment by jkarneges

I've had no trouble hitting the Firebase API at the speed items are created, with a 5 second delay between retries.

For scraping HN directly, in my experience you have to go extremely slow, like 1 minute between fetching items. And if you get blocked, it may be better to wait a long time (minutes) before trying again rather than exponential backoff, in order to get out of the penalty box. You'll need a cache for sure.

Comment items look like https://hacker-news.firebaseio.com/v0/item/45533616.json?pri...:

  {
    "by" : "jkarneges",
    "id" : 45533018,
    "kids" : [ 45533616 ],
    "parent" : 45532549,
    "text" : "The HN&#x2F;Firebase API doesn&#x27;t make this easy. For <a href=\"https:&#x2F;&#x2F;hnstream.com\" rel=\"nofollow\">https:&#x2F;&#x2F;hnstream.com</a> I ended up   crawling items to find the article.",
    "time" : 1760043552,
    "type" : "comment"
  }

"parent" can either be the actual parent comment or the parent article, depending where in the comment chain you are.