Proxy performance benchmarking usually fails for a boring reason: it measures something nobody actually runs. A single-threaded request to one search engine through one exit tells you almost nothing about how a pool behaves when forty workers pull product pages through it at 09:00. The numbers look clean, the provider looks fine, and then production falls over on a Tuesday.

A benchmark worth trusting starts from your workload and works backward: same request mix, same targets, same concurrency, same time of day, and percentiles instead of averages. This guide covers the metrics that matter, the sample sizes that make them meaningful, and the comparison rules that keep two providers honest.

Start from the workload, not from the tool

  • Request mix. Record what fraction of your traffic is HTML, JSON, images and file downloads. A pool tuned for small JSON calls behaves very differently on 20 MB transfers.
  • Target set. Use five to ten real domains you care about, not one search engine. Different targets sit behind different TLS stacks, bot defences and CDN edges.
  • Concurrency. Test at the parallelism you actually run, typically 10 to 50 workers. Serial testing flatters every provider equally and hides queueing entirely.
  • Session model. State whether each worker holds a sticky session or rotates per request, because that one choice changes success rate more than the provider does.

Proxy performance benchmarking checklist: six numbers to collect

  1. Latency p50 per target — the typical experience, useful but never sufficient on its own.
  2. Latency p95 and p99 over at least 500 samples per endpoint — the tail is where timeouts, retries and abandoned jobs live.
  3. Success rate at 10, 30 and 50 parallel workers, counting only responses that match the expected status and payload, not merely a TCP connect.
  4. DNS path: socks5 versus socks5h — remote resolution adds a hop but removes a leak, and often shifts measured latency by 20–80 ms in either direction.
  5. Throughput on a fixed 10–50 MB payload, repeated three times to separate a warm cache from a cold transfer.
  6. Churn: exit IP changes per hour on a sticky session. Anything above the advertised lifetime is a broken promise worth writing down.

Comparison

Three metrics carry most of the decision, provided they are measured together rather than in isolation:

MetricWhat it revealsHow to collect it
p50 latencyTypical page experience and baseline overhead500+ requests per target, timings from your client, not the dashboard
p95 / p99 latencyTail pain, retry pressure, worker idle timeSame run, percentiles computed per endpoint, outliers logged
Success rate at 10–50 parallelReal capacity before bans and queueingExpected status plus payload check, failures bucketed by reason

Read the three metrics as a set. A provider with p50 of 620 ms, p95 of 900 ms and 97% success at 30 workers is usually the better production choice than one with p50 of 480 ms, p95 of 4,100 ms and 88% success, even though the second wins the headline number. The tail and the failure rate decide how much retry logic, queueing and supervision your pipeline needs, and those costs land on your team rather than on the invoice. Keep the raw logs too: provider behaviour changes after infrastructure work, and a benchmark without the underlying samples cannot be re-analysed when someone asks why the pool felt slower last month.

Run each test long enough for rare failures to appear. A one-minute burst at 30 workers produces about 300 requests, which is far too few to observe a 2% ban rate; ten minutes gets you into the thousands and exposes the pattern behind the percentage. Watch for failures clustering on one subnet or one exit, because a provider with 99% success concentrated in a single bad block behaves very differently from one with 99% spread evenly. Both look identical in a summary table and behave nothing alike in production.

How to benchmark a proxy (steps)

  1. Freeze the environment. Same machine, same client version, same region, same script for every provider. A laptop on hotel Wi-Fi invalidates the whole exercise.
  2. Warm up, then discard. Throw away the first 50 requests: TLS handshakes, DNS caches and connection pools are still cold and will distort the tail.
  3. Run two windows a day. Residential pools breathe with their users, so a 03:00 result and a 14:00 result can differ by 40%. Report both instead of picking the flattering one.
  4. Repeat three times and report medians. Averages get dragged by one 30-second outlier; the median of three runs survives a noisy host.
  5. Log failures with a reason. Separate timeouts, 403s, 407 auth errors and TLS failures, because each points at a completely different fix.
  6. Test at production concurrency. Push to the parallel count you actually run before you sign a contract, not after the first incident.

Common mistakes

  • Benchmarking one endpoint. A provider that wins on a search engine can lose badly on a marketplace with aggressive bot defence.
  • Reporting averages. The mean hides exactly the requests that break your pipeline; p95 and p99 are the numbers your users feel.
  • Ignoring churn. A sticky session that silently rotates every few minutes destroys logged-in workflows while still looking healthy on paper.
  • Comparing across different hours or days. If provider A ran at 03:00 and provider B at 16:00, you measured the internet, not the providers.

FAQ

Why does p95 matter more than p50?

The p50 tells you what a typical request costs. The p95 tells you what happens to one request in twenty, and at 50 workers that is a timeout every few seconds. Retries then inflate load, which pushes the tail further out — the failure compounds. If p50 improves while p95 worsens, the pool got worse for automation.

How many samples do I need?

At least 500 per endpoint for percentiles, and more if you intend to quote p99, because that percentile is estimated from the last few samples. For success rates, 1,000 requests gives roughly plus or minus 3 percentage points at 95% confidence — enough to separate a good pool from a bad one.

Why does proxy speed vary so much between runs?

Because a residential pool is a shared, living system. Peer machines sleep, ISPs re-route, targets deploy new defences, and your own upstream link competes with everything else in the office. This is why datacenter vs residential comparisons must control for time and target, and why a single fast run proves nothing.

What operational metrics matter beyond the raw numbers?

Replacement speed when an exit dies, how quickly the dashboard shows a bad node, whether failures are attributed to a subnet or a single IP, and whether support answers with a fix or a template. A provider 20 ms slower on p50 but ten times faster at replacing dead exits is usually the better buy.

Benchmarking is cheap; discovering your pool fails at 09:00 on launch day is not. Build the harness once, keep the same targets and concurrency, and rerun it quarterly and after every provider change. If you want a baseline to compare against, FTProxy residential proxies publishes per-node latency and success metrics and gives you the same endpoints we test.