How to migrate from Firecrawl to String in Python (2026)

Last updated: October 19, 2026. Benchmark figures come from the September 16, 2026 run of the Web Data Frontier Benchmark. The String code on this page ran on October 15, 2026, and the output shown is what it printed. The Firecrawl calls ran on October 19, 2026 on Firecrawl's keyless free tier, with only the API key left out. We build String, the API this page migrates to.

To migrate from Firecrawl to String in Python, replace each firecrawl.scrape(url, formats=["markdown"]) call with one POST to https://request.usestring.ai/v1/fetch that sends {"url": url, "format": "markdown"}, then read the site's own status code from the x-status-code response header. The parameter table in Step 2 maps the rest.

You do not have to move everything at once. Keep Firecrawl and send only its failures to String. In our September 16, 2026 benchmark run, that fallback moved 25 of 100 sites and raised the share of attempts that returned the page from 80.2% to 97.2%.

This page is part of our Firecrawl alternatives comparison. For the head-to-head numbers, see String vs Firecrawl.

On this page: TL;DR · Where String fits · What people run into on Firecrawl · What a fallback changes on 100 sites · Step 1 to Step 7 · How to avoid getting blocked after the move · How String handles a Firecrawl migration · What it costs · FAQ

TL;DR

• The core swap is one function. firecrawl.scrape(url, formats=["markdown"], only_main_content=True) becomes a POST to /v1/fetch with "format": "markdown" and "mainContentOnly": true.
• Two defaults flip. Firecrawl strips page chrome by default (onlyMainContent: true) and String keeps it (mainContentOnly: false). Firecrawl skips TLS verification by default and String verifies certificates.
• Firecrawl's /crawl returns page content. String's /v1/sitemap returns the URLs, and you fetch each one. A 10-page discovery job on our own blog cost $0.003 on October 15, 2026.
• Keeping Firecrawl and rerouting only the sites it did not return on all five attempts moved 25% of benchmark attempts to String and lifted coverage to 97.2%, against 97.0% for String alone and 80.2% for Firecrawl alone.
• On the 17 sites where Firecrawl returned nothing on any attempt, String returned 14 on all five attempts.

Where String fits

String fits one job in this migration: getting the page back when Firecrawl does not.

• For: Python teams that call Firecrawl's scrape endpoint for markdown or raw HTML, see blocks or empty results on protected sites, and want a second provider without rewriting their pipeline.
• Not for: teams that depend on Firecrawl's agent and extract endpoints, page-change monitoring, document parsing uploads, or a self-hosted open-source crawler. Keep Firecrawl, or its open-source core, for those jobs and send String the fetches.
• Price: you pay only when content comes back. String Growth is $100 a month, plus $0.20 per 1,000 requests on a plain fetch with standard proxies, $2.00 with premium proxies, $1.00 for a browser fetch and $4.00 for a browser fetch with premium proxies. Starter is $20 a month at $0.30, $3.00, $1.50 and $6.00. The first 5,000 standard requests are free (pricing). The nearest Firecrawl plan at that price, Standard, is $99 a month billed monthly for 100,000 credits (Firecrawl pricing).
• Integrations: a REST API you call with requests, an MCP server for agents, n8n and Zapier integrations, and String Actors on Apify Store.
• Limits: 60 requests a second per organization with a 3,600-request burst, and no cap on concurrent requests. Firecrawl Standard allows 25 concurrent browsers and 500 scrape requests a minute.

What people run into on Firecrawl

The first thing to know argues against switching. Plenty of Firecrawl failures are fixed inside Firecrawl. In January 2026 a user reported SCRAPE_ALL_ENGINES_FAILED on a product page, and a Firecrawl team member answered with a setting:

"looks like this website has a delay when rendering the data, you can add a wait action to ensure you get all data back." abimaelmartell, Firecrawl issue #2691, January 20, 2026

Try that first. If a wait action fixes the page, you do not need a second provider for it.

The harder failure is a block page that arrives as a success. In November 2025 a cloud user wrote that a scrape was "marked as succeeded in activity logs and eats a credit," while the HTML was Cloudflare's 5xx error page (mcarreon, Firecrawl issue #2413). They added that a retry with stealth mode failed once and then worked. Firecrawl staff replied that the request had used its more expensive fire-1 engine and offered a refund. The lesson for any migration: check the body as well as the status. Step 4 does that.

The third failure is specific to the Python SDK and to batch jobs:

"batch_scrape() and crawl() return failed and cancelled jobs to the caller exactly like successful ones" nainilnandu-gush, Firecrawl issue #4212, August 3, 2026

The same report says no exception is raised and the returned object carries no error field. The reporter verified it on firecrawl-py 4.34.0. If your pipeline calls batch_scrape(), check each job's status before you trust an empty result. The replacement in Step 5 returns failures as visible errors.

What a fallback changes on 100 sites

The Web Data Frontier Benchmark sends the same 100 URLs to 16 scraping APIs, five attempts each. A load passes only when the API returns a 2xx status and the body contains text from the real page, so a challenge page served at 200 counts as a failure. We asked one question of the per-site results: if a Firecrawl user keeps Firecrawl and sends only the sites Firecrawl did not return on all five attempts to String, what changes?

The reroute moves 25 sites, 125 of 500 attempts. On those 25 sites Firecrawl returned 26 of 125 attempts and String returned 111. Firecrawl returned nothing on 17 of them, and String returned 14 of those 17 on every attempt.

The rerouted sites are not random. Six are social networks (Reddit, LinkedIn, Instagram, Facebook, TikTok and Pinterest). Five are retailers, among them Home Depot, Walmart and Best Buy, and four are fashion sites. If your Firecrawl traffic is mostly docs and blogs, your reroute share will be lower. If it is retail or social, it will be higher.

Six sites stayed below five of five after the reroute, including temu.com, which no API in the run returned. The raw result file and the harness are public, so you can rerun the split for your own site list.

Two caveats. Firecrawl ran through /v2/scrape, asking for raw HTML, with its default proxy setting, which retries on enhanced proxies when basic ones are blocked, and with its cache off (maxAge: 0). With its default two-day cache, your own traffic may score differently. And not every provider in the run used its strongest anti-bot mode; the Firecrawl adapter shows exactly what we sent. The 25% figure also assumes one page per site, so weight it by your own traffic.

Try the reroute on your own failures first: take the ten URLs that fail most in your Firecrawl logs and run Step 3 on them with a free key. The first 5,000 standard requests cost nothing.

Step 1: list every Firecrawl call

A migration is as big as the set of Firecrawl features you use. Find them before you change anything:

grep -rnE "firecrawl|Firecrawl\(|\.scrape\(|\.crawl\(|\.map\(|\.search\(|batch_scrape|start_crawl" --include=*.py .

Write down, for each call: the method, the formats you ask for, every option you pass, and what your code reads back (doc.markdown, doc.html, doc.metadata.status_code, doc.json). Then pull the 20 domains you scrape most. Those domains decide whether the migration is easy, because blocking is per site.

Step 2: map each parameter

Every row below comes from Firecrawl's API reference and String's fetch reference, both read on October 15, 2026. "No direct equivalent" means keep Firecrawl for that job or build it yourself.

Sources: Firecrawl scrape reference, proxies and Python SDK; String fetch reference, response formats, browser actions and sitemap reference.

Step 3: replace the scrape call

This is the Firecrawl call most Python projects make. It follows the Python SDK docs. We ran it on October 19, 2026 on the SDK's keyless free tier: it returned status 200 and 9,329 characters of markdown that included the product title.

import os
from firecrawl import Firecrawl  # pip install firecrawl-py

firecrawl = Firecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])

doc = firecrawl.scrape(
    "https://www.homedepot.com/p/Milwaukee-M18-FUEL-18-Volt-Lithium-Ion-Brushless-Cordless-Gen-II-18-Gauge-Brad-Nailer-Tool-Only-2746-20/309752194",
    formats=["markdown"],
    only_main_content=True,
)
print(doc.metadata.status_code, len(doc.markdown))

This is the String version. It keeps the same inputs and returns the same two things, the markdown and the site's status code:

import os
import requests

STRING_URL = "https://request.usestring.ai/v1/fetch"


class FetchFailed(Exception):
    pass


def string_scrape(url, main_content_only=True, country=None, timeout=90):
    """Return (markdown, status_code), like firecrawl.scrape(url, formats=["markdown"])."""
    payload = {"url": url, "format": "markdown", "mainContentOnly": main_content_only}
    if country:
        payload["countryCode"] = country  # Firecrawl: location={"country": "US"}
    response = requests.post(
        STRING_URL,
        headers={"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"},
        json=payload,
        timeout=timeout,
    )
    if response.status_code != 200:
        # A non-200 here is String's own error (key, rate limit, unreachable site), not the page's.
        raise FetchFailed(f"String {response.status_code}: {response.text[:200]}")
    status = int(response.headers.get("x-status-code", 0))  # the site's own status
    return response.text, status

We ran it on three URLs on October 15, 2026: Firecrawl's own introduction page, the Home Depot product page above and a DigiKey product page. Home Depot and DigiKey are both sites Firecrawl returned on none of five attempts in the September run.

docs.firecrawl.dev/introduction   200   14,783 chars   0.64 s
homedepot.com (brad nailer)       200   25,944 chars   2.87 s
digikey.com (NE555P)              200  182,447 chars  13.93 s

The Home Depot markdown opened with the product title, "Milwaukee M18 FUEL 18-Volt Lithium-Ion Brushless Cordless Gen II 18-Gauge Brad Nailer (Tool-Only) 2746-20". The DigiKey markdown opened with "NE555P Texas Instruments".

Run this with a free key: pip install requests, export STRING_API_KEY=..., then call string_scrape() on your own URLs.

Step 4: check the status and the body

A Firecrawl response and a String response each carry two statuses: the API's and the site's. Keep them apart. On String, a non-200 from /v1/fetch is String's own error, and x-status-code is what the site answered. A site's 404 comes back inside a 200 from String.

Then check the body. The Cloudflare page in issue #2413 arrived as a success, and a block page can arrive as a 200 from any provider. This check catches the common cases:

BLOCK_MARKERS = ("cf-error-details", "Just a moment...", "captcha-delivery", "Access Denied", "px-captcha")


def looks_blocked(markdown, status, must_contain=None):
    """A 200 is not proof of content. Check the status, the size, block markers and one expected string."""
    if status >= 400 or len(markdown) < 500:
        return True
    if any(marker in markdown for marker in BLOCK_MARKERS):
        return True
    return bool(must_contain) and must_contain not in markdown

Pick must_contain by copying text from the real page. We first passed "Loewe" for a Saks Fifth Avenue product page and the check failed, because the page spells it "LOEWE". The check is case-sensitive on purpose. With "Striped Slim Zip-Front Top" it passed. Our page on detecting a block page that returns HTTP 200 covers more markers.

If your logs show blocked pages today, this is the place to test String: point string_scrape() at those URLs with a free key and run looks_blocked() on what comes back.

Step 5: port JSON extraction, search, map and batch

JSON extraction. Firecrawl takes the schema inside formats: formats=[{"type": "json", "schema": Product.model_json_schema()}], and returns it as doc.json. String takes the same JSON Schema as jsonSchema with "format": "json":

schema = {
    "type": "object",
    "properties": {"title": {"type": "string"}, "price": {"type": "number"}, "brand": {"type": "string"}},
    "required": ["title", "price"],
}
response = requests.post(
    "https://request.usestring.ai/v1/fetch",
    headers={"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"},
    json={"url": HOME_DEPOT_URL, "format": "json", "jsonSchema": schema},
    timeout=120,
)
print(response.headers.get("x-json-schema-applied"), response.json())

On October 15, 2026 it printed true and {"title": "Milwaukee M18 FUEL 18-Volt Lithium-Ion Brushless Cordless Gen II 18-Gauge Brad Nailer (Tool-Only) 2746-20", "price": 329, "brand": "Milwaukee"} in 2.1 seconds. When extraction cannot satisfy the schema, String returns the unextracted page with "extracted": false and a reason, so check x-json-schema-applied before you trust the object.

Search. Firecrawl's firecrawl.search() can scrape each result in the same call. String's /v1/search returns ranked results, and you fetch the ones you need:

response = requests.post(
    "https://request.usestring.ai/v1/search",
    headers={"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"},
    json={"query": "firecrawl python sdk scrape markdown", "searchCount": 5},
    timeout=60,
)
for result in response.json()["results"]:
    print(result["rank"], result["url"])

It returned five results in 0.97 seconds, the first being docs.firecrawl.dev/sdks/python.

Map and crawl. Firecrawl's map() returns URLs and crawl() returns pages. String splits discovery from fetching. You submit a job, get a price quote, approve it, then read the URLs:

import time

API = "https://request.usestring.ai/v1"
H = {"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"}

job = requests.post(f"{API}/sitemap", headers=H, timeout=60,
                    json={"url": "https://usestring.ai/blog", "maxPages": 10, "maxDepth": 1, "pathPrefix": "/blog"}).json()
print(job["estimatedCostUsd"], job["estimatedPages"])  # nothing runs or bills until you approve
requests.post(f"{API}/sitemap/{job['jobId']}/approve", headers=H, timeout=60)
while True:
    status = requests.get(f"{API}/sitemap/{job['jobId']}", headers=H, timeout=60).json()
    if status["status"] not in ("running", "awaiting_approval"):
        break
    time.sleep(5)
urls = requests.get(f"{API}/sitemap/{job['jobId']}/urls", headers=H, params={"limit": 1000}, timeout=60).json()["urls"]
pages = [string_scrape(u["url"]) for u in urls]  # the crawl() half

The job on our blog finished in 7.9 seconds with 10 URLs, quoted at up to $0.053 and settled at $0.003. Quotes expire after an hour, and budgetUsd caps any job.

Batch scrape. String has no batch endpoint and no concurrency cap, so batch_scrape() becomes a thread pool. Each failure comes back as an error with its URL:

from concurrent.futures import ThreadPoolExecutor


def batch_scrape(urls, workers=20):
    def one(url):
        try:
            markdown, status = string_scrape(url)
            return {"url": url, "status": status, "markdown": markdown}
        except FetchFailed as error:
            return {"url": url, "status": None, "error": str(error)}
    with ThreadPoolExecutor(max_workers=workers) as pool:
        return list(pool.map(one, urls))

Three of our own blog pages came back in 0.7 seconds, each with status 200. Keep workers under the account's 60 requests a second.

Step 6: keep Firecrawl and route failures to String

This is the step that changes the least code. Firecrawl stays the primary. String gets a URL only when Firecrawl raises an error or returns a page that fails looks_blocked():

def firecrawl_scrape(url):
    from firecrawl import Firecrawl
    firecrawl = Firecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])
    doc = firecrawl.scrape(url, formats=["markdown"], only_main_content=True)
    return doc.markdown or "", doc.metadata.status_code or 0


def scrape(url, must_contain=None):
    """Return (markdown, provider). Firecrawl first, String when Firecrawl fails or returns a block page."""
    try:
        markdown, status = firecrawl_scrape(url)
        if not looks_blocked(markdown, status, must_contain):
            return markdown, "firecrawl"
    except Exception as error:  # SDK errors, timeouts, SCRAPE_ALL_ENGINES_FAILED
        print(f"firecrawl failed on {url}: {error}")
    markdown, status = string_scrape(url)
    if looks_blocked(markdown, status, must_contain):
        raise FetchFailed(f"both providers failed on {url} (last status {status})")
    return markdown, "string"

We tested the String half by making the Firecrawl half raise on purpose. On October 15, 2026, the Saks Fifth Avenue product page came back from String in 0.46 seconds with the product name present, and the Instacart Costco storefront in 3.4 seconds with "Costco" present. Both are sites Firecrawl returned on none of five attempts in the September run.

Log the provider value. After a week, the share of URLs that went to String is your own reroute rate, and it tells you whether to stop at a fallback or move everything.

Step 7: run both side by side before you cut over

Before you change the primary, run both providers on the same sample and compare. Take 200 URLs from last week's logs, weighted to your top 20 domains, and record per URL: Firecrawl's status and size, String's status and size, and whether looks_blocked() passed for each.

Three numbers decide the cutover. The share of URLs each provider returned, the share where only one did, and the median size difference on URLs both returned. A large size gap on the same URL usually means one side returned page chrome and the other did not; check mainContentOnly and markdownMode before you blame the provider.

Your parsers may need one change. String's markdown keeps structured data from the page as fenced JSON blocks, and rewrites relative links to absolute ones. If you parse Firecrawl's markdown with regular expressions, run them on String's output in this sample first.

How to avoid getting blocked after the move

A new provider does not change how sites decide to block you. The same rules apply.

• Check the body on every response. Step 4's looks_blocked() is the most important code on this page.
• Pace each domain. Firecrawl's delay crawl option had a reason to exist. Space requests to one site, and cache pages you fetch twice, as String's rate-limit docs advise.
• Keep custom headers on direct fetches. Custom headers cannot be combined with a browser fetch on String, and a mismatched User-Agent can make a block more likely.
• Set a country only when the page needs one. Prices and stock change by country, so set countryCode for those pages and leave it off elsewhere.
• Read the failure. A 403 and a 429 need different fixes. Our pages on fixing a 403 and handling rate limits cover both, and how to scrape without getting blocked covers the rest.

How String handles a Firecrawl migration

String is the most reliable web scraping API, bypassing anti-bot, and you only pay when content comes back. For a team moving off Firecrawl, that means:

• One request replaces one scrape call. POST /v1/fetch with "format": "markdown" returns the page as markdown, and String picks a plain request, a premium proxy or a real browser per site (fetch docs).
• The path that ran is in a header. The x-billed-request-type header names it. On October 15, 2026 our JSON extraction of the Home Depot page billed as request_premium.
• String returned 97.0% of attempts on 100 sites. In the September 16, 2026 run, String returned 485 of 500 attempts, the highest of 16 APIs (benchmark).
• String returned 14 of the 17 sites Firecrawl returned on no attempt. Each of the 14 came back on all five attempts in the same run.
• A blocked request is not billed. String charges only when content comes back (pricing).
• There is no concurrency cap. Throughput is set by 60 requests a second with a 3,600-request burst, on every plan (pricing).

What it costs

Firecrawl prices in credits. On Standard, $99 a month billed monthly buys 100,000 credits; a basic scrape is 1 credit a page, and JSON extraction adds 4 credits a page. Firecrawl does not charge a scrape that returns no result, and a page that answers 403 or 404 still costs 1 credit (Firecrawl pricing, billing docs).

String prices per successful request, at the rate of the path the site needed. For a fallback that sends String 25,000 requests a month (25% of 100,000 pages), usage on each path would be:

Each total assumes every request took that one path. A real month mixes them, and requests that come back blocked cost nothing. The rates come from usestring.ai/pricing on October 15, 2026; JSON extraction adds a fee that scales with page and schema size.

Start with the URLs that fail in your Firecrawl logs today. Run Step 3 on them with a free key, and you will know your reroute rate before you change a line of production code.

FAQ

How do I migrate from Firecrawl to another web scraping API?

List every Firecrawl call and option you use, map each one to the new API, swap the scrape call, and add a body check. Then run both providers on a sample of your own URLs before you cut over. With String, the scrape call becomes one POST to /v1/fetch with "format": "markdown", and the site's status moves to the x-status-code header.

What is the String equivalent of Firecrawl's scrape endpoint?

POST https://request.usestring.ai/v1/fetch. Send "format": "markdown" for markdown, "format": "raw" for raw HTML, or "format": "json" with jsonSchema for structured data. Set "mainContentOnly": true to match Firecrawl's default of stripping navigation and footers.

Can I use Firecrawl and String together?

Yes. Keep Firecrawl as the primary and send String only the URLs Firecrawl fails on, as in Step 6. In the September 16, 2026 benchmark run, that sent String 25 of 100 sites and raised coverage from 80.2% of attempts to 97.2%.

What are the best alternatives to Firecrawl?

That depends on what you use Firecrawl for. Our Firecrawl alternatives comparison ranks the alternatives on the same benchmark run, and String vs Firecrawl compares the two head to head. Open-source options such as Crawl4AI suit teams that want to host the crawler themselves.

Is there a free version of Firecrawl?

Yes. Firecrawl's free plan gives 1,000 credits a month with no card, and its Python SDK can scrape and search without a key on a rate-limited free tier. String's first 5,000 standard requests are free, which is enough to test a fallback on your own failing URLs.

Why does Firecrawl return a Cloudflare page or an "all scraping engines failed" error?

SCRAPE_ALL_ENGINES_FAILED means every engine Firecrawl tried failed on that URL; for slow-rendering pages, Firecrawl staff recommend a wait action. A Cloudflare page means the site blocked the request and the block page came back as content. Check the body for block markers on every response, retry, and route repeat failures to a second provider.

Does Firecrawl charge for failed requests?

Firecrawl does not charge a scrape that returns no result, and a page that answers with an error status such as 403 or 404 costs 1 credit, per its pricing page on October 15, 2026. In one 2025 report, a Cloudflare error page came back as a successful scrape and used a credit, and Firecrawl offered a refund. String bills only when content comes back.

How do I replace Firecrawl crawl and map with String?

Use POST /v1/sitemap for discovery. It returns a price quote, starts after you approve it, and returns the URLs it found; then fetch each URL with /v1/fetch. A 10-page job on our own blog finished in 7.9 seconds and cost $0.003 on October 15, 2026.

Sources

• String's Web Data Frontier Benchmark, September 16, 2026 run: per-target cells for Firecrawl and String on 100 sites, five attempts each. Raw results and Firecrawl adapter in the public harness.
• String test runs, October 15, 2026: /v1/fetch in markdown and JSON formats, /v1/search, /v1/sitemap, a thread-pool batch, and the String half of the fallback, from Python 3.9 with requests.
• Firecrawl, scrape API reference, crawl, map, search, batch scrape, JSON extraction, proxies, Python SDK, billing and pricing, all read October 15, 2026.
• Firecrawl GitHub issues #2691 (January 2026), #2413 (November 2025) and #4212 (August 2026).
• String, fetch reference, response formats, browser actions, sitemap reference, search reference, rate limits and pricing, October 15, 2026.