Scrape ecommerce competitor prices on Lazada, Shopee, and Amazon

A residential proxy pool gets around 80% success on Shopee SG. A mobile carrier pool gets 95% or higher. That 15 percentage point gap compounds across millions of requests a month into a noticeably cleaner dataset and a noticeably smaller proxy bill.

Ecommerce price intelligence means monitoring competitor prices, stock levels, promotions, and assortment across marketplaces and brand sites at high frequency. In 2026 this is harder than it was three years ago. Anti-bot detection has matured aggressively, especially on Amazon, Shopee, Lazada, Tokopedia, JD, and Mercado Libre. Mobile proxies have become the operational baseline for high-volume pipelines because their detection profile beats residential and ISP IPs on the platforms that matter most.

This article covers three things: the five concerns a working pipeline handles, the five anti-bot layers and which ones mobile proxies actually fix, and a working Shopee SG scraper pattern with rotation and S3 archival.

What price intelligence requires

Five things, repeatedly, at scale.

Product discovery: maintain a current catalog of competitor SKUs across categories, mapped back to your own SKUs. Price scraping: collect current price, list price, sale price, and promotion details at intervals ranging from hourly to weekly. Stock and availability tracking: detect stockouts, low-stock warnings, shipping windows, and restocking patterns. Promotion and bundle detection: flash sales, BOGO offers, percentage discounts, voucher stacks, and free shipping thresholds. Assortment monitoring: new product launches, discontinuations, and category expansions.

Each of these requires hundreds to millions of HTTP requests per day depending on category breadth. Each request goes to a marketplace with a sophisticated anti-bot stack. Detection failures lead to blocked IPs, captcha walls, or stale data. The IP infrastructure you choose determines which of those you get.

Why mobile beats residential on Shopee and Lazada

Datacenter IPs: low to medium success, often blocked, very low cost, fits internal QA. Residential IPs: medium to high success, medium to high cost per gigabyte, fits most public marketplace scraping. ISP static IPs: medium to high success, sticky sessions, fits sticky workloads. Dedicated mobile IPs: high success, priced per port as a flat fee, fits logged-in scraping and mobile-first marketplaces.

For Singapore marketplaces where mobile-first apps dominate consumer behavior, mobile proxies meaningfully outperform residential because the anti-bot fingerprints expect mobile-carrier ASN traffic. The Shopee app, the Lazada app, and the Carousell app are mobile-first products. Their bot detection is calibrated for mobile-carrier patterns. Hitting them from a residential desktop pool means fighting the wrong fingerprint.

The five anti-bot layers

IP reputation. Datacenter IPs get scored down at the first request. Residential IPs pass unless flagged. Mobile IPs share CGNAT with thousands of real users and are very hard to blocklist without collateral damage. Mitigation: use mobile or residential IPs.

TLS and HTTP fingerprinting. Python’s default requests library has a distinctive TLS fingerprint (a JA3 hash) that differs from a real browser’s. Some marketplaces blocklist non-browser TLS. Mitigation: use libraries like curl_cffi or tls_client that mimic browser TLS, or use a real browser via Playwright.

Behavioral analysis. Rapid sequential requests, no mouse movement, no scrolling, no realistic dwell times, all stand out. Mitigation: pace requests, randomize timing, and add realistic browsing patterns.

Cookie and session state. Many marketplaces require a warm session before serving full product data. Cold requests get partial data. Mitigation: maintain session state per IP and warm up before scraping.

Device fingerprinting. Canvas, WebGL, font list, screen resolution, and hardware concurrency are all visible to JS execution. Mitigation: use a real browser with anti-detect features.

Mobile proxies address the first layer. The other four require additional engineering on top of the IP.

A working scraper

The rough shape: rotate through a pool of mobile ports, fetch a category or SKU page, check the response status, and hand off good HTML to a parser while bad responses trigger a rotation. Production code adds retry queues, circuit breakers per host, response deduplication, parser version tracking, and observability hooks on top of that shape.

Rotation patterns

Fixed-interval rotation: rotate every N minutes regardless of activity. Simple and predictable. It wastes some IP freshness but works well for high-volume scraping. Bad-status rotation: rotate on a 429, a 403, or a captcha response. This preserves IP freshness but needs good circuit breaker logic. Session-aware rotation: rotate at the boundary between scraping sessions, which fits workloads that need short-lived stable sessions.

For Shopee SG and Lazada SG, fixed-interval rotation every 15 to 30 minutes works well combined with bad-status rotation as a fallback. For Carousell, longer intervals of 1 to 2 hours work better, since the platform is less aggressive on rate limiting.

Storage and re-scrapability

The most important architectural decision is storing raw HTML responses durably. Anti-bot signatures change. Marketplaces redesign category pages. Your parsers will fail occasionally. When they do, you want to re-parse historical responses without re-scraping, since re-scraping costs proxy bandwidth and risks rate limits.

Store responses in S3 or GCS with content-addressed paths and metadata indexing. A typical schema uses year-month-day-hour folders by marketplace, then a file named after the SKU hash with an html.gz extension. Add a parquet index per day with SKU, marketplace, fetch timestamp, content hash, and status code. The parquet index lets you query historical fetches without listing object storage directly. Gzipping the HTML keeps storage cost modest.

Singapore-specific tactics

Scraping Shopee SG, Lazada SG, and Carousell from a Singapore mobile carrier IP gives you a meaningfully different experience than scraping from residential or datacenter IPs. The marketplace serves Singapore-localized listings, prices in SGD, shipping options to Singapore addresses, and Singapore-specific promotions. The JS bundles include local market features. The anti-bot scoring is calibrated to expect mobile-carrier traffic from local users.

The practical impact: a single mobile port scraping Shopee SG, Lazada SG, and Carousell at moderate frequency typically delivers 90 to 95% success without hitting aggressive captcha walls. The same volume from a residential pool typically delivers 75 to 85%. That difference compounds across millions of requests.

If you want to see this in practice, point a scraper at Shopee SG, Lazada SG, or Carousell using a dedicated Singapore mobile IP and compare the success rate against your existing residential or datacenter pipeline.

Ready to run your own comparison? Singapore Mobile Proxy gives you dedicated Singtel, StarHub, and M1 mobile IPs for scraping work like this.

Get new guides and videos first — join the Telegram channel.