Two numbers decide how a Wellfound run behaves: how many requests are in flight, and what kind of address they come from. Both were measured rather than guessed.
The limit follows the IP
Wellfound rate-limits per IP address. There is no account involved in this Actor at all, so the only thing a limit can attach to is the address the requests come from — which is why the proxy choice is not a performance tweak but the main variable.
Datacentre against residential
| Proxy type | Requests | Result |
|---|---|---|
| Datacentre | 36 | 24 returned HTTP 429 |
| Residential | 48 | 48 returned HTTP 200 |
Same concurrency, same targets. Two thirds failing against none. That is why residential is the default in the input schema rather than a line of advice in a README.
Where the concurrency ceiling is
16 in flight is the highest value measured without a block, and the input caps there for that reason. 24 triggers a 429 with roughly a five-minute cooldown. The default is 6, which leaves headroom rather than running at the measured edge — a limit that held today is not a limit that holds forever.
Why exceeding it makes a run slower
A blocked request is not free. It costs a round trip, it returns nothing, and the cooldown that follows stops the requests that would have succeeded. Past the ceiling, raising concurrency trades successful requests for failed ones at a bad exchange rate.
What to actually set
Leave the proxy on residential. Leave concurrency at 6 unless a run is time-critical, and do not exceed 16. Cap the run with maxJobs — not with pages per role.
