In updating the 192 public and private repos for Netflix (likely also due the large number of issues on some projects), what I see is most get processed in about 16 minutes and then the rest an hour later. I believe this is due to the github access library just looping for an hour waiting for the rate limits to reset. While this works, it isn't the best. It would be better to just quit once rate limits are exhausted as the scraper is written to be be run multiple times and we, at Netflix, run it three times per day.
In updating the 192 public and private repos for Netflix (likely also due the large number of issues on some projects), what I see is most get processed in about 16 minutes and then the rest an hour later. I believe this is due to the github access library just looping for an hour waiting for the rate limits to reset. While this works, it isn't the best. It would be better to just quit once rate limits are exhausted as the scraper is written to be be run multiple times and we, at Netflix, run it three times per day.