ActorStack.dev
Scraper engineering

Rate limiting

Also: HTTP 429, throttling

A server-side control that rejects requests once a client exceeds a threshold, usually with HTTP 429 and a cooldown before normal service resumes.

The cooldown is what makes aggressive concurrency counterproductive: measured against Nextdoor, concurrency 5 ran clean while 10 triggered a roughly four-minute penalty, which costs more time than the extra parallelism saves.

Where this comes up