Last updated: September 13, 2026. Benchmark figures come from the August 11, 2026 run of the Web Data Frontier Benchmark. Pricing was checked against each vendor's published pages on September 13, 2026.
The best web scraping tools in 2026 are String, Scrapfly, Context.dev, Firecrawl, Bright Data and Oxylabs if you want a hosted web scraping API, Apify if you want a prebuilt scraper for a named site, Crawl4AI, Scrapy or Playwright if you want to run the code yourself, and Octoparse if you do not write code. String returned content on 97.0% of requests across 99 bot-protected sites, the highest of 15 APIs tested. Scrapfly placed second at 82.0%, Context.dev third at 79.2% and Firecrawl fourth at 78.6%. The open-source frameworks and Octoparse were not tested, because they are not APIs a harness can call the same way. This page compares all eleven on what they do, what they cost, and where each one fails.
The choice starts with who carries the blocking problem. An open-source framework is free and you handle proxies, browsers and captchas yourself. A hosted API charges per page and handles them for you. The benchmark measures only the second group, so every rank on this page is an API rank.
| Tool | Type | Best for | Benchmark success | Price near $100 a month | The catch |
|---|---|---|---|---|---|
| String | Hosted API | Protected sites that must return content | 97.0% | Growth, $100: $0.20 per 1,000 plain fetches, $2.00 premium | No crawl-to-content endpoint and no SOC 2 report yet |
| Scrapfly | Hosted API | Several APIs on one key | 82.0% | Pro, $100: 1,000,000 credits; $0.10 to $2.50 per 1,000 pages | Slow on pages it shares with String; credits do not roll over |
| Context.dev | Hosted API | One flat credit per page | 79.2% | No $100 plan; Pro, $149: $0.745 per 1,000 pages | Weak on Kasada at 33.3% |
| Firecrawl | Hosted API | Crawl-to-content and speed | 78.6% | Standard, $99 billed monthly: $0.99 per 1,000 pages | A page that returns a 403 document still costs a credit |
| Bright Data | Hosted API | Proxies and datasets on one contract | 78.0% | No plan near $100; $1.50 per 1,000 successful requests | Slowest latency score in the run |
| Oxylabs | Hosted API | PerimeterX targets | 76.8% | Web Unblocker Micro, $75 for 8GB | Billed by bandwidth, so no per-page rate |
| Apify | Scraper marketplace | A prebuilt scraper for a named site | Not tested | Starter $19 or Scale $199, plus usage | Each Actor has its own price and its own maintainer |
| Crawl4AI | Open-source crawler | LLM-ready Markdown on your own servers | Not tested | Free, Apache-2.0 | No proxies or unblocking included |
| Scrapy | Open-source framework | Large crawls you control in Python | Not tested | Free, BSD-3-Clause | Does not render JavaScript on its own |
| Playwright | Open-source browser automation | Pages that need clicks and scripts | Not tested | Free, Apache-2.0 | A browser without a proxy pool is easy to block |
| Octoparse | No-code desktop app | Point-and-click scraping without code | Not tested | Standard, $119 billed monthly | Tasks and concurrency are capped by plan |
Benchmark figures come from the August 11, 2026 run: 99 sites, five attempts each, 495 requests per API. The latency score is failure-aware: per site, the p75 of an API's successful attempts, and a site an API never returned inherits the p75 of the APIs that did. Prices come from each vendor's published pages, checked September 13, 2026, at the plan closest to $100 a month billed monthly.
There are four kinds, and they differ on one question: who handles blocking.
A hosted web scraping API takes a URL and returns the page. Proxy rotation, browser rendering and captcha handling happen on the vendor's side, and you pay per page or per credit. String, Scrapfly, Context.dev, Firecrawl, Bright Data and Oxylabs are this kind. It is the right choice when your sites are protected and your team would rather not run proxy infrastructure.
A scraper marketplace sells ready-made scrapers for specific sites. Apify is the example here, with more than 68,000 Actors in its store. It is the right choice when someone has already built and maintains a scraper for the exact site you need.
An open-source framework is code you run. Scrapy crawls at scale, Playwright drives a real browser, and Crawl4AI turns pages into Markdown for language models. They cost nothing to license. They are the right choice when your sites are unprotected, or when you already own proxies and want full control.
A no-code app lets you click on the data you want. Octoparse is the example here. It is the right choice for a non-developer with a small, steady list of sites.
Many teams end up with two kinds: a framework or no-code app for easy sites, and an API for the ones that block.
We ran 99 live sites, five attempts each, against 15 APIs, for 495 requests per API. Each API ran through one adapter published in the repository. The timeout was 90 seconds. A request passed only when the response contained a marker from the real page, so a captcha page returning HTTP 200 counted as a failure. The 99 sites sit behind DataDome, Akamai, Cloudflare, PerimeterX, AWS WAF, Kasada, Fastly and in-house systems.
Two adapters on this page changed after the run. On September 6, 2026 the harness was changed to start Scrapfly on its residential pool with rendering and Firecrawl on its enhanced proxy. The August 11 run predates that change and has not been repeated, so the Scrapfly and Firecrawl figures describe the lighter setting that ran. The String, Context.dev, Bright Data and Oxylabs adapters were not part of that change.
Scrapy, Playwright, Crawl4AI and Octoparse were not tested. They are not hosted services, so their success depends on the proxies and code you bring. Apify was not tested because a marketplace has no single scraper to call. None of them gets a rank on this page.
Disclosure: String built this benchmark and sells the API that ranks first in it. The answer to that conflict is a harness you can run yourself. The targets, pass rule and every adapter are open on GitHub and accept your own keys. The original benchmark post explains why the latency score refuses to reward fast failures. Providers contest results and we publish the corrections: Firecrawl's engineer opened a pull request against its adapter, and the rerun raised its score by 7.7 points.
String's Web Access API makes one HTTP call against any URL. Proxy rotation, anti-bot handling, captcha solving and JavaScript rendering happen behind the endpoint.
Pricing. Usage-based. Growth costs $100 a month and bills $0.20 per 1,000 fetches on a standard proxy, $2.00 on a premium proxy, $1.00 for a browser request on a standard proxy, and $4.00 with both. Starter costs $20 a month at $0.30, $3.00, $1.50 and $6.00. The first 5,000 standard requests are free, blocked requests are not billed, and credits roll over.
Benchmark success. 97.0%, or 480 of 495, first of 15. All five attempts on 92 sites, partial content on 5, nothing on 2 (temu.com and skyscanner.net).
Latency. A latency score of 9.98 seconds, second of 15 behind Firecrawl.
Strengths. 100% against Cloudflare, Kasada and Fastly, 98.0% against Akamai and 97.9% against DataDome. A hosted MCP server gives an agent fetch, search and crawl without local setup.
Weaknesses. No crawl-to-content endpoint, no self-hosting, no prebuilt site scrapers and no SOC 2 report, where Scrapfly and Firecrawl both hold one. Plain fetches are cheap, but a premium-proxy page costs more than Context.dev's flat credit.
Pricing. Pro is $100 a month for 1,000,000 credits, with overflow at $3.50 per 10,000. A datacenter scrape is 1 credit, rendering or the Unblocker is 5, and residential proxies are 25, so a page costs $0.10, $0.50 or $2.50 per 1,000. Failed requests are free. Credits do not roll over. The free tier is 1,000 credits with no time limit.
Benchmark success. 82.0%, or 406 of 495, second of 15. Nothing returned on 8 sites, the fewest after String.
Latency. A latency score of 18.35 seconds. On the 66 sites where Scrapfly and String both passed every attempt, Scrapfly's median was 11.75 seconds against String's 3.55.
Strengths. Scraping, browser, crawler, extraction and screenshot APIs share one key and one credit pool, and it ships a hosted MCP server. It holds SOC 2 Type II, ISO 27001 and HIPAA attestations. It was the only API besides String to pass 100% of the real-estate sites.
Weaknesses. Speed on pages both return, and a bill that moves with the setting: its anti-bot mode can move a request to the residential pool, which is 25 times the datacenter rate.
Pricing. Developer is $25 a month for 10,000 credits and Pro is $149 for 200,000, so there is no plan at $100. Pro works out at $0.745 per 1,000 pages, with overage at $9 per 10,000 credits. One credit is one scrape, with proxies, rendering and anti-bot bypass included. Browser actions cost 2 credits. Failed or blocked requests are not billed. The free tier is 1,000 credits a month.
Benchmark success. 79.2%, or 392 of 495, third of 15. All five attempts on 73 sites, the most of any API after String.
Latency. A latency score of 12.68 seconds, fourth of 15.
Strengths. The most predictable bill here, because a protected page costs the same credit as an easy one. 100% on social media, jobs and ticketing sites, and an MCP server.
Weaknesses. 33.3% against Kasada and 66.7% against AWS WAF, and nothing returned on 16 sites. The plan ladder skips $100.
Pricing. Standard is $99 a month billed monthly for 100,000 credits, or $0.99 per 1,000 pages; the page defaults to the annual view at $83. A scrape that returns no document is free, but a page that responds with a 403 or 404 is returned and costs a credit. Credits roll over only on Scale. The free tier is 1,000 credits a month.
Benchmark success. 78.6%, or 389 of 495, fourth of 15, on the lighter setting described in the method section.
Latency. A latency score of 9.21 seconds, the best of all 15 APIs.
Strengths. Crawl, map, search and scrape endpoints built for LLM input, an open-source core under AGPL-3.0, SOC 2 Type II, and an MCP server. 100% against Cloudflare and 87.3% on fashion and luxury sites, the best of the five other APIs.
Weaknesses. 59.2% on in-house protection systems and 25% on social media, and nothing returned on 16 sites. See our Firecrawl alternatives page for the full breakdown.
Pricing. No plan near $100. Pay as you go is $1.50 per 1,000 successful requests, and the Scale tier is $499 a month for 383,000 requests. Premium domains bill at a higher rate shown only in the dashboard. The free tier is 5,000 requests a month.
Benchmark success. 78.0%, or 386 of 495, fifth of 15. The largest partial middle of the six APIs: 31 sites returned some attempts and not others.
Latency. A latency score of 26.14 seconds, the slowest of 15.
Strengths. The broadest catalogue here, with proxies, a browser API, SERP APIs and datasets, plus a free MCP server. 93.3% against AWS WAF.
Weaknesses. Speed, 60.0% against Akamai, and a premium-domain rate that is not published, so it is never named the cheapest option on this page.
Pricing. Web Unblocker is billed by bandwidth. Micro is 8GB for $75 a month, Starter 38GB for $325, and Advanced 88GB for $660. Code WU40 takes 40% off for six months. The free trial is 1GB, which Oxylabs describes as up to 10,000 results.
Benchmark success. 76.8%, or 380 of 495, sixth of 15.
Latency. A latency score of 14.67 seconds.
Strengths. 97.8% against PerimeterX, the best of all 15 APIs, and 96% on grocery sites. An MCP server is on GitHub.
Weaknesses. Bandwidth billing makes heavy pages expensive and forecasts hard. 50.0% against AWS WAF, 25% on ticketing, and nothing on the one Fastly site. See our Oxylabs alternatives page.
Pricing. Free includes $5 of monthly usage, Starter is $19 a month and Scale is $199, each as prepaid usage plus pay as you go. Compute runs $0.20 per unit on Starter and $0.16 on Scale, and residential proxies cost $8 per GB on Starter. Unused prepaid usage expires at the end of each billing cycle. Store Actors charge either per event or per usage, and each Actor's page says which.
Benchmark success. Not tested. A marketplace has no single scraper to benchmark.
Strengths. The Apify Store lists more than 68,000 Actors, so a scraper for a named site often already exists. It offers scheduling, storage, an MCP integration, and Crawlee, its open-source crawling library. It shows a SOC 2 badge on its trust centre.
Weaknesses. Each Actor is maintained by its own developer, so quality and upkeep vary, and a site redesign can break one. Costs depend on run time and retries, which makes a bill hard to predict before a test run.
Pricing. Free. Crawl4AI is open source under Apache-2.0 and runs on your own machines. Any LLM extraction you add bills through your own model provider.
Benchmark success. Not tested. Its results depend on the proxies and browser settings you supply.
Strengths. Built to turn pages into clean Markdown for RAG and agents, with a self-hosting guide and a large, active community on GitHub. No per-page fee at any volume.
Weaknesses. No proxy pool and no unblocking service are included, so a DataDome or Akamai site blocks it the way it blocks any headless browser. You also run and patch the infrastructure.
Pricing. Free. Scrapy is open source under BSD-3-Clause.
Benchmark success. Not tested.
Strengths. A long-established Python crawling framework, with asynchronous request scheduling, auto-throttling, item pipelines and middlewares, as its overview describes. It is fast on sites that serve plain HTML and it scales to very large crawls.
Weaknesses. It does not run JavaScript by default. Its own documentation on dynamically loaded content points to parsing embedded data or adding a headless browser. It has no proxies or captcha handling, so protected sites need an API or proxy service behind it. Several APIs on this page, including Scrapfly, publish Scrapy integrations for that reason.
Pricing. Free. Playwright is Microsoft's open-source browser automation framework under Apache-2.0.
Benchmark success. Not tested.
Strengths. One API drives Chromium, Firefox and WebKit, as its getting-started guide shows. It handles clicks, logins, scrolling and single-page apps, and it can intercept network requests to read the JSON a page loads.
Weaknesses. It is a testing tool first. A browser alone brings no residential IPs and no fingerprint management, so protected sites detect and block it. At scale you run and pay for every browser.
Pricing. A free plan includes 10 tasks and 50,000 exported rows a month. Standard costs $119 a month billed monthly, or $83 on a lower-usage tier, and Professional costs $299. The $69 and $249 figures in Octoparse's FAQ are the annual rates, which save 16%. Paid plans carry a 5-day money-back guarantee.
Benchmark success. Not tested.
Strengths. A desktop app where you click the data you want, with 500+ preset templates on Standard. Standard includes IP rotation, residential proxies, captcha solving and cloud runs.
Weaknesses. Standard allows 100 tasks and 3 concurrent cloud processes, and Professional 250 tasks and 20. Click-built tasks break when a site changes its layout, and the app does not suit a pipeline that needs code review or version control.
Success rates by protection vendor for the six APIs, from the August 11 run. Site counts are in the column headers.
| API | DataDome (19) | Akamai (20) | Cloudflare (15) | PerimeterX (9) | AWS WAF (6) | Kasada (3) | Fastly (1) | In-house (26) |
|---|---|---|---|---|---|---|---|---|
| String | 97.9 | 98.0 | 100 | 88.9 | 96.7 | 100 | 100 | 96.2 |
| Scrapfly | 81.1 | 71.0 | 92.0 | 82.2 | 83.3 | 80.0 | 80.0 | 85.4 |
| Context.dev | 74.7 | 83.0 | 90.7 | 73.3 | 66.7 | 33.3 | 100 | 82.3 |
| Firecrawl | 89.5 | 72.0 | 100 | 84.4 | 76.7 | 93.3 | 100 | 59.2 |
| Bright Data | 82.1 | 60.0 | 85.3 | 68.9 | 93.3 | 93.3 | 100 | 81.5 |
| Oxylabs | 76.8 | 67.0 | 77.3 | 97.8 | 50.0 | 93.3 | 0 | 83.8 |
Among the five other APIs, each one leads at least one column: Firecrawl on DataDome and Cloudflare, Context.dev on Akamai, Oxylabs on PerimeterX, Bright Data on AWS WAF, and Scrapfly on in-house systems. That spread is why no one of them covers a mixed list.
Every page in this series carries one statistic computed from the run's per-site results. For a page about tools, the question is how many you have to buy.
Take the five other APIs on this page: Scrapfly, Context.dev, Firecrawl, Bright Data and Oxylabs. Count the sites where at least one of them returned all five attempts, for the best possible combination at each size.
String alone returned all five attempts on 92 of 99 sites. Two of the other APIs together fall short of that. Three pass it by three sites, and four pass it by five. The best result at each size grows by less each time: 73, then 17 more, then 5, then 2.
The four-tool stack is real coverage, and it beats one API. It also means four integrations, four sets of failure handling, and four bills. At the plans nearest $100 a month, the three-tool stack costs $348 in plans before usage (Scrapfly Pro $100, Context.dev Pro $149 and Firecrawl Standard $99), and the four-tool stack $423 with Oxylabs Micro. You also need routing code that knows which API to send each site to, and that code breaks when a site changes protection.
Two sites defeated every combination: temu.com, which also shut out String, and coupang.com, where String returned four of five. The five sites the stack returns fully that String does not are mouser.com, skyscanner.net, neimanmarcus.com, allegro.pl and booking.com. String returned four of five on four of those and nothing on skyscanner.net.
For an agent that must fetch live pages, a hosted API with an MCP server is the shortest path, because the agent calls one tool and gets the page back. String passed 97.0% of requests on the August 11, 2026 benchmark and ships a hosted MCP server; Scrapfly, Context.dev, Firecrawl, Bright Data and Oxylabs also publish MCP servers. Crawl4AI is the open-source option when you can run your own infrastructure and your sites are not heavily protected.
For a list of bot-protected sites, more than one API can help, but each one adds less. On our 99-site benchmark the best single API other than String returned all five attempts on 73 sites, the best pair on 90, the best three on 95, and the best four on 97. String alone returned 92. The three-tool stack costs $348 a month in plans at the $100 anchor, before usage and routing code.
Octoparse is the established no-code option. Its free plan includes 10 tasks and 50,000 exported rows a month, and Standard costs $119 a month billed monthly, with residential proxies and captcha solving included. We did not benchmark it. Its limits are task counts and concurrency, and click-built tasks need fixing when a site changes its layout.
They answer different problems. Scrapy, Playwright and Crawl4AI are free and give full control, and they work well on sites that do not block automation. A paid API earns its price on protected sites, because it supplies the residential proxies, browser fingerprints and captcha handling a framework leaves to you. On our benchmark's 99 protected sites, the six APIs on this page returned between 76.8% and 97.0% of requests.
Switch, or add an API behind Scrapy, when your block rate rises above what retries fix, or when your team spends more time on proxies and captchas than on the data. Scrapy does not render JavaScript by default and ships no proxies. Many teams keep Scrapy for crawling logic and send only the protected requests to an API.
A scraping API returns pages you request, and your code decides what to extract and when. A managed service, such as Bright Data's managed data acquisition or a custom Octoparse crawler, builds and runs the collection for you and delivers the data. Choose an API when you have engineers and a changing list of sites. Choose a managed service when you want a fixed dataset delivered and no code to maintain.
String gives the first 5,000 standard requests free. Context.dev and Firecrawl each give 1,000 credits a month, Scrapfly gives 1,000 credits with no time limit, and Bright Data gives 5,000 requests a month. Apify's free plan includes $5 of monthly usage and Octoparse has a free plan with 10 tasks. Scrapy, Playwright and Crawl4AI are free open-source software.
Yes. The harness, the 99 sites, the pass rule and every adapter are open on GitHub, and it runs with your own API keys. Per-site results for all 15 APIs are on the benchmark page, and our three-way comparison of ScraperAPI, ZenRows and ScrapingBee shows how to test a shortlist on your own list.
Checked September 13, 2026. Benchmark figures: August 11, 2026 run, official_results/benchmark-2026-08-11T22-44-25-322Z.json in the harness repository.
Benchmark data
Hosted APIs
Marketplace, frameworks and no-code
