Proxy pool management is the difference between a stack that quietly degrades on a Friday afternoon and one that pages you before customers notice. A pool is only as good as its worst working IP: one rotten exit inside a rotation window can poison an entire job, while dozens of idle-but-healthy IPs inflate your bill without contributing anything.
Teams that treat their pool as infrastructure — monitored, scored and versioned like any other production system — consistently outperform teams that treat it as a text file of hostnames. This guide lays out the operating loop: health checks, scoring, cooldown, rotation policy and failure attribution. Each part feeds the next — probes produce the data scores are built from, scores drive routing, routing outcomes become the attribution log, and the log refines tomorrow’s cooldown and retirement decisions.
What proxy pool management actually runs on
- Continuous health checks: a background worker probes every IP every few minutes for connect latency, TLS handshake success and DNS behavior, so you learn about failures before your jobs do.
- Separating dead from sick: a dead IP refuses connections entirely; a sick one connects but returns errors, captchas or soft blocks. They need completely different handling.
- Scoring instead of guessing: every IP carries a live score built from success rate, latency and recency of last block, and traffic is routed by that score.
- Attribution after every response: the exit IP is logged per request, so any failure can be answered with “which IP, which error, which retry”.
The six-point proxy pool management checklist
- Probe on a schedule: ping all IPs every few minutes; record latency percentiles, not just up/down flags.
- Classify failures: dead (no TCP/TLS) goes to retirement review; sick (errors, captchas, 403 storms) goes to cooldown and re-test.
- Score every IP: success rate over the last N requests, rolling average latency, and hours since the last block — three numbers beat one.
- Route by score: critical traffic only to IPs above your threshold; exploratory jobs may try unknowns and let the scoreboard learn.
- Enforce cooldown in hours: a blocked or captcha-served IP rests for hours, not minutes, before re-testing; repeat offenders are retired with a quarantine log entry.
- Log attribution: exit IP per response, task ID, retry count — this is the dataset your scoring and alerting are built on. Store it long enough to answer questions like “did this ASN decay after a specific date”, because that is exactly what a recurring failure will eventually ask.
Comparison
Each IP in a pool lives in one of four states, and the right response differs by state:
| IP state | Typical signals | Correct action |
|---|---|---|
| Healthy | Fast connect, clean responses, no blocks for weeks | Keep serving critical traffic; monitor drift |
| Sick | Connects but errors, captchas or soft 403s appear | Cooldown for hours, then re-test with low-value traffic |
| Dead | No TCP connect or repeated TLS handshake failure | Immediate quarantine; replace from reserve stock |
| Retired | Repeated cooldown cycles, score below the floor | Remove from rotation permanently; log the pattern |
The state machine matters more than any single metric. Most incidents are not one dead IP but a transition nobody noticed — a healthy IP quietly sliding into sick, or a cooldown population growing faster than replacements arrive.
How to set up proxy pool management step by step
- Inventory the pool: import every endpoint into one place with provider, country, session type and purchase date as metadata.
- Stand up the prober: one worker, one probe definition (connect, TLS, sample request), one results table — boring and reliable beats clever.
- Pick scoring inputs: start with success rate over the last 50 requests, p95 latency, and hours since last block; weight success rate highest.
- Define routing rules: critical jobs pull only from IPs above a score threshold; batch jobs may use everything; sticky jobs pin one IP for the session.
- Set cooldown and retirement policy: for example six hours of cooldown after a block, retirement after three blocks in 14 days, always with a log line explaining why.
- Wire alerting: page when the healthy-IP count drops below your safety margin or when a task type’s success rate dips — static ISP pool sizing tells you how much reserve to keep. Two alert levels are enough to start: a warning at 80% of the margin and a page at 60%, with task-type success rate watched separately.
Common mistakes
- Rotating mid-transaction: login → cart → checkout must share one identity; the rotating vs sticky guide covers which tasks tolerate rotation at all.
- Treating sick IPs as dead: throwing away a captcha-tripped IP that one cooldown would have healed burns budget for no reason. The reverse error costs just as much: keeping a sick IP in the critical rotation because the dashboard still says “up”.
- Retiring without a quarantine log: without patterns (ASN, country, time of day) you keep buying the same bad ranges.
- Sizing the pool on the average afternoon: plan for peak concurrency plus probe overhead plus reserve, not your median Tuesday.
FAQ
How big should a proxy pool be?
Size from peak concurrency, task duration and acceptable reuse: pool ≈ concurrency × (session length ÷ reuse interval) × safety factor of 1.5–2. Add the reserve that static ISP pool sizing recommends — usually 20–30% — so a ban wave never stalls production. When in doubt, oversize slightly and let scoring route the surplus to exploratory work: an idle healthy IP costs cents, a stalled pipeline costs hours.
How do I score IPs fairly?
Use a rolling window, not lifetime averages: success rate over the last 50–100 requests, p95 latency, and hours since last block. Lifetime stats let one historically good IP hide a week of decay; rolling stats catch the decay while it is still cheap to fix.
When should an IP be retired instead of cooled down?
Retire when blocks repeat inside the cooldown window or the score stays under your floor after two full re-tests. One block is noise; three in two weeks is a reputation problem no cooldown fixes. Always retire with a reason code in the quarantine log — after a quarter you will want to know whether retirements clustered around one provider, one country or one week.
What should alerting cover?
Three layers: infrastructure (probe failures, exit-IP churn), pool health (healthy-IP count, cooldown population) and outcome (success rate per task type). Outcome alerts matter most — they are what the business actually feels.
Proxy pool management is a loop, not a purchase: probe, classify, score, route, cool down, attribute, repeat. It needs no exotic tooling — a scheduler, a results table and discipline beat most commercial dashboards. Run it weekly and the pool stops being a source of surprises. FTProxy residential proxies expose per-node health and usage in the dashboard, so half of this loop is already built in.