Amazon scraping with proxies is not really a speed problem — it is a geography and pacing problem. The same ASIN shows a different price, a different buy-box winner and sometimes a different availability depending on where the request originates, so an IP that merely rotates fast produces data that is wrong in a way you cannot see.
This guide covers what Amazon personalizes by IP, how to pin residential IPs per marketplace and region, when rotation helps and when it destroys accuracy, and the pacing and snapshot discipline that keeps a catalog pipeline alive for months rather than days. The rules are unglamorous, but they are the difference between a pipeline that survives a quarter and one that dies in a week.
What Amazon personalizes by geography
- Price and currency: conversion, tax display and promotional pricing shift by marketplace domain and often by detected region inside a country. Two requests for the same ASIN can legitimately return two totals.
- Buy-box ownership: which seller wins the Buy Box is decided per request using delivery location, seller performance and stock. A residential IP in the wrong region can hand the box to a seller your customers never see.
- Availability and delivery promises: “in stock”, delivery dates and Prime eligibility are computed from the destination implied by the IP. A US crawl from a European exit reports the wrong delivery promise.
- Reviews and content variants: review populations, “customers also viewed” rails and localized titles differ by marketplace, and partial localization appears even within one domain.
Six-point checklist before the first crawl
- Enumerate your targets as marketplace plus region pairs — amazon.com/US-East, amazon.com/US-West, amazon.de, amazon.co.jp — not just domains. Region is where pricing logic actually lives.
- Reserve a pool of geo-pinned residential IPs per pair, sized for your concurrency and for the number of catalogs you track. Our notes on static ISP pool sizing explain how to size without overbuying.
- Decide per page type whether you rotate or stay sticky. Search pages tolerate rotation; product detail pages do not, because a mid-session IP change can alter the offer set mid-capture.
- Set Accept-Language, currency cookie and a plausible timezone per marketplace. Amazon reads all three, and a Japanese locale on a German IP produces a hybrid page that matches no real user.
- Choose a random delay between 5 and 15 seconds per request, with occasional longer pauses. Uniform fast bursts are the single most reliable way to get throttled.
- Snapshot a timestamp, the exit IP and the region with every captured offer. Without them, an intraday price change is indistinguishable from a bug.
Comparison
| Page type | Proxy strategy | Why it matters |
|---|---|---|
| Search results | Rotation per request within one region pool | High request volume, low state; rotating spreads load without corrupting a session |
| Product detail | Sticky IP for the whole capture, 5–15 minutes | Price, buy-box and delivery promise must come from one consistent identity |
| Reviews | Sticky, with the same IP reused for pagination | Review pagination depends on session state; rotating mid-list returns duplicates |
| Best Sellers and Deals | Per-request rotation, sampled at fixed hours | Rankings move; consistent sampling times matter more than IP variety |
How to build the pipeline, step by step
- Start with one marketplace and one region. Get price, buy-box and delivery fields stable before adding a second geography — otherwise you cannot tell a parsing bug from a regional difference.
- Route through a geo-pinned IP and verify the exit country and ASN before the request. See how to test a proxy for the validation calls worth automating.
- Send marketplace-appropriate headers and cookies: language, currency, and a session that persists for the duration of a detail capture.
- Apply randomized pacing of 5 to 15 seconds, plus exponential backoff with jitter on 429. On 503 or a CAPTCHA interstitial, rotate the IP and cool that target down for several minutes rather than retrying immediately.
- Parse defensively: prices arrive in multiple layouts, buy-box blocks vary, and some pages render a fallback template. Store the raw HTML fragment for every field so a parser regression is recoverable.
- Write one row per offer per capture, with capture time, exit IP, region, currency and seller. Deduplicate on ASIN plus region plus timestamp bucket, and keep the raw snapshot for audit.
Common mistakes
- Treating marketplace as country. amazon.com serves several regions with different prices; pinning to “US” and calling it done quietly mixes them.
- Rotating during a product capture. The price, buy-box and delivery promise can each resolve against a different identity, producing an offer that never existed.
- Uniform request rates. A perfectly even stream of requests is a fingerprint. Randomize the interval and let it drift.
- Overwriting snapshots. Keeping only the latest price destroys the intraday history that repricing analysis depends on.
FAQ
Why does Amazon block my scraper even with proxies?
Usually because the behavior, not the IP, is the trigger: uniform intervals, identical header sets across thousands of requests, or a fingerprint that contradicts the claimed device. Proxies only change the network identity. Fix pacing, vary headers plausibly, keep cookies coherent within a session, and reduce concurrency before buying more IPs — concurrency is the cheapest lever to test and usually the one that was set too high.
Which proxy type works best for Amazon?
Residential IPs pinned to the marketplace region. Datacenter ranges are widely known and often served a degraded experience — a different layout or a stripped offer set — which corrupts the data even when the request succeeds. Residential is also what makes buy-box and delivery promises resemble what a real shopper sees.
How do I collect reviews without duplicates?
Keep one sticky IP per review crawl and paginate within that session, because the review list depends on session state. Deduplicate on review ID rather than position, and capture the “top reviews” and “most recent” views separately since Amazon orders them differently. Re-crawl on a schedule and diff by ID to pick up edits and removals.
Is Amazon scraping legal?
It depends on jurisdiction, the data you collect and how you use it. Public product data is generally treated differently from personal data, and terms of service still apply even where scraping is not itself unlawful. Collect only what you need, avoid personal information, respect rate limits, and get legal review for your specific case rather than relying on a blog post.
Conclusion
Amazon rewards slow, geographically consistent collection and punishes fast, uniform collection. Pin residential IPs per marketplace and region, rotate only on stateless pages, pace at five to fifteen seconds, and snapshot every offer with its timestamp and exit IP. Start with FTProxy residential proxies and treat the region field as a first-class part of your schema, not an afterthought.