Best API to scrape LinkedIn and other protected sites

What is the best API to scrape LinkedIn and other protected sites?

The best API to scrape LinkedIn and other protected sites returns the public page on every attempt, and lets you tell a real page from a sign-in wall. In String's September 16, 2026 benchmark run, 4 of 16 APIs returned all eight social sites on all five attempts each: String, Scrapfly, Bright Data and Browserbase, 40 of 40 requests. String also had the fastest average LinkedIn time in the field, 1.7 seconds.

On October 28, 2026 we fetched 21 public LinkedIn URLs three times each through String. 57 of 63 came back as the real public page. The other 6 were school pages, where LinkedIn served its sign-in wall with a 200 status. String builds the benchmark and ranks itself in it. The harness and the per-site results are public.

What 16 APIs returned on eight social sites

Eight of the 100 benchmark pages are social networks. The Web Data Frontier Benchmark sent each API five requests to each page on September 16, 2026.

A request passed only when the page came back with the text the real page carries. A 200 with a challenge page did not count.

* These eight APIs ran in the September 16 run on a setting below the anti-bot mode they offer today. Their social numbers may understate what they return now. The benchmark labels every social site's defense as proprietary: each network runs its own.

Four results stand out.

• Social sites are easier than they look. All 16 APIs together returned 78.1% of social requests, against 66.6% across all 100 sites.
• LinkedIn split the field. 11 of 16 APIs returned LinkedIn on all five attempts. Three returned it on none: Firecrawl, Scrapingdog and Zyte.
• A high overall score does not carry over. Firecrawl returned 80.2% of all requests and 10 of 40 social requests. Browserbase returned 41.4% overall and 40 of 40 here.
• Four APIs dropped on social pages. ScraperAPI, Scrapingdog, Firecrawl and ZenRows each scored lower on these eight sites than on all 100.

One caveat on the LinkedIn column. The benchmark's LinkedIn page is Microsoft's company page, and its pass check is the word "Microsoft" in the body. A sign-in wall can carry that word too. So we ran a stricter test of our own, below.

Run one LinkedIn page through String now: curl -X POST https://request.usestring.ai/v1/fetch -H "Authorization: Bearer $STRING_API_KEY" -H "Content-Type: application/json" -d '{"url":"https://www.linkedin.com/company/stripe/"}'. The first 20,000 standard fetches are free.

What 63 LinkedIn fetches returned on October 28

57 of 63 String fetches returned LinkedIn's real logged-out page on October 28, 2026. We sent 21 public URLs three times each: 8 company pages, 4 job search pages, 4 job postings, 3 public profiles of well-known executives and 2 school pages.

We judged each response by LinkedIn's own pageKey meta tag, which names the page LinkedIn chose to serve, plus the text that page type must carry.

All 63 responses carried status 200. A check on the status code alone would have scored 63 of 63. The 6 school responses were LinkedIn's sign-in wall, served on purpose to every logged-out visitor.

• No request needed a premium proxy. 53 of 63 took a plain standard fetch and 10 a standard browser.
• Median time was 1.2 seconds per request, and the slowest took 6.9 seconds.
• 19 of 21 URLs came back real on all three attempts. The two that did not were the school pages.

That is the main lesson for any LinkedIn-style site. Getting a response is not the hard part. Knowing which page you got is.

Where String fits

• For: teams that read public company pages, job listings and job postings on LinkedIn and other social networks, for hiring signals, market maps, competitor tracking or lead research, and need every scheduled check to come back.
• Not for: anything behind a login, such as full search results, Sales Navigator, connections, groups or messages. String does not sign in to LinkedIn and does not hold accounts. It is also not for buying or reselling member data.
• Price: on the Starter plan ($19 a month), a plain fetch is $0.25 per 1,000 on standard proxies and a browser fetch is $1.20 (pricing). On October 28 every LinkedIn request took the standard path. String bills when it brings back the site's response, so a sign-in wall is billed like any page. Leave school URLs out of your list.
• Access: social and professional networks may need approval for your organization before the first request. Until then the API returns a 403 that names the domain (access control docs). Email support with your use case.
• Integrations: a REST API you call from any language, a hosted MCP server for agents, n8n and Zapier integrations, and LangChain and AI SDK guides.
• Limits: 15 requests a second on Starter, up to 90+ on Volume, and up to 10,000 concurrent fetches (pricing). String fetches public pages only.

How String returns LinkedIn pages

String is the most reliable web scraping API, bypassing anti-bot, and you only pay when content comes back. Here is what that means for LinkedIn-style sites, one fact per line:

• String returned all eight social sites on every attempt. It passed 40 of 40 requests to LinkedIn, Instagram, Facebook, X, TikTok, Pinterest, Reddit and YouTube in the September 16, 2026 run, and 97.0% of 500 requests across all 100 sites, the highest of 16 APIs (benchmark).
• String had the fastest LinkedIn time of 16 APIs. It averaged 1.7 seconds on LinkedIn in the September 16 run.
• One call picks the route each page needs. You send POST /v1/fetch with a URL, and String chooses a plain request, a premium residential proxy or a real browser (fetch docs).
• The response names the route. On October 28, the x-billed-request-type header read request_standard on 53 of 63 LinkedIn requests and browser_standard on 10.
• The page comes back whole, so you can check it. String returns LinkedIn's HTML with its pageKey tag intact, so your code can drop a sign-in wall before it parses anything.
• Fields come back as JSON on request. Send a JSON Schema in jsonSchema and the response is the extracted object, such as company name, industry, size and follower count (extraction docs).

Why LinkedIn-style sites are hard

LinkedIn guards member data with both technology and lawsuits. In its case against Proxycurl, a LinkedIn data API, the company's lawyers said the scrapers' fake accounts are usually

"detected by LinkedIn's technical defenses and restricted within about a day"

LinkedIn's complaint, as quoted by Bloomberg Law, December 2025

Proxycurl shut down in July 2025 instead of fighting the case, and LinkedIn sued a second data company, ProAPIs, that October (Bloomberg Law). Both cases centered on fake accounts that scraped behind the login.

That sets the line for a scraping API. Logged-out public pages are one job. Data behind the sign-in wall is a different job, with a different legal risk.

The second difficulty is quiet failure. An agency operator on r/automation described a LinkedIn tool that had stopped for four days before anyone saw it:

"things break by going quiet and staying quiet."

u/Silly_Captain2089, r/automation, 2026

A sign-in wall with a 200 status is the same failure in a smaller form. Your job reports success, and your table fills with empty rows.

Python: fetch a LinkedIn page and check what came back

This script sends each URL to String's fetch endpoint and judges the result by LinkedIn's page type, not the status code.

import os
import re
import sys

import requests

API = "https://request.usestring.ai/v1/fetch"
HEADERS = {"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"}

# Page types LinkedIn serves to logged-out visitors, by its own pageKey meta tag
PUBLIC_PAGES = (
    "d_org_guest_company_overview",
    "d_jobs_guest_search",
    "d_jobs_guest_details",
    "public_profile",
)


def fetch(url: str) -> dict:
    r = requests.post(API, headers=HEADERS, json={"url": url, "format": "json"}, timeout=120)
    r.raise_for_status()
    body = r.json()
    html = str(body.get("data", ""))
    key = re.search(r'<meta name="pageKey" content="([^"]+)"', html)
    page_key = key.group(1) if key else ""
    return {
        "status": body.get("statusCode"),  # LinkedIn's own status: 200 even on a sign-in wall
        "page_key": page_key,
        "real_page": page_key.startswith(PUBLIC_PAGES),  # judge by page type, not status
        "sign_in_wall": "auth_wall" in page_key,
        "path": r.headers.get("x-billed-request-type"),  # route String used
    }


if __name__ == "__main__":
    for url in sys.argv[1:]:
        print(url, fetch(url))

We ran it on October 28, 2026 with pip install requests on three URLs. The Stripe company page and the data engineer job search came back with real_page: True on a standard fetch. MIT's school page came back with status 200, page_key: auth_wall_desktop_school and sign_in_wall: True.

For a hiring-signal job, point it at company pages and job search URLs. For other networks, swap the pageKey check for text the real page must carry, as the benchmark does.

What public LinkedIn pages carry, and what they do not

A logged-out company page carries more than most people expect. On October 28, Stripe's page carried its industry, company size, headquarters, founding year, website and follower count.

Job search pages carried links to individual postings, and each posting carried its seniority level and employment type. The executive profile we read in full carried a work history.

What a member's public profile shows depends on that member. LinkedIn honors each member's choices about who can see their profile (User Agreement, section 2.5), so an ordinary member's public page can carry far less than an executive's. Plan for gaps.

Profiles also hold personal data. Our page on scraping personal data under GDPR and CCPA covers what the law asks of you.

Public pages, terms and the law

Reading a logged-out public page is not the same act as scraping behind a login. LinkedIn's User Agreement forbids members to use crawlers "to scrape or copy the Services" (section 8.2), and it binds you once you accept it.

The courts have drawn the line elsewhere for public data. Our page on the hiQ v. LinkedIn ruling explains what that case decided and what it left open. The cases that hit scrapers hardest involved fake accounts and logins, as with Proxycurl.

This page covers reading public pages. It does not cover logged-in data, fake accounts or reselling member profiles. This is not legal advice; our page on whether web scraping is legal goes further.

The boundary

String is one of four APIs that returned all eight social sites on every attempt in the September 16 run. Scrapfly, Bright Data and Browserbase matched it on these pages, and Scrapfly did so on a setting below its current anti-bot mode.

If your list is LinkedIn company pages only, 11 of 16 APIs returned the benchmark's LinkedIn page on all five attempts. Then speed, price and how well you can spot a sign-in wall decide. LinkedIn-specific data APIs that sell parsed profiles were not tested in the benchmark.

Start with the LinkedIn URLs your job misses today. Run the script above on them with a free key, and read the real_page and sign_in_wall fields before you parse a single row.

FAQ

Best API to scrape LinkedIn-style protected sites without getting blocked?

In String's September 16, 2026 benchmark run, String, Scrapfly, Bright Data and Browserbase were the only 4 of 16 APIs to return all eight social sites on all five attempts each, LinkedIn included. String also had the fastest LinkedIn time, 1.7 seconds. String builds the benchmark; the harness is public.

Can you scrape LinkedIn without getting blocked?

Logged-out public pages, yes. On October 28, 2026, 57 of 63 String fetches of public LinkedIn URLs returned the real page, with no premium proxy. The 6 misses were school pages, where LinkedIn serves a sign-in wall to every logged-out visitor.

Why does my LinkedIn scraper get status 200 but no data?

LinkedIn serves its sign-in wall with status 200. In our October 28 test, all 63 responses carried status 200, and 6 of them were sign-in walls. Check the page's pageKey meta tag, or text the real page must carry, instead of the status code.

Which LinkedIn pages can you scrape without logging in?

On October 28, 2026, company pages, job search pages, job postings and the public profiles we tried all came back as real pages to a logged-out request. School pages returned a sign-in wall. Search results, connections, groups and messages need a login.

Is it legal to scrape LinkedIn?

Reading public pages and scraping behind a login are different acts. LinkedIn's User Agreement forbids scraping by members, and LinkedIn has sued data companies that used fake accounts, such as Proxycurl, which shut down in July 2025. This is not legal advice.

Do I need residential proxies to scrape LinkedIn?

Not for logged-out public pages in our test. On October 28, 2026, String routed all 63 LinkedIn requests on standard proxies: 53 as a plain fetch and 10 in a standard browser. String moves a request to a premium proxy only when a site blocks the standard one.

What is the best Proxycurl alternative?

Proxycurl shut down in July 2025 after LinkedIn sued it. If you need public company pages and job listings, a scraping API that returns them reliably is one route: String returned LinkedIn on every benchmark attempt. Parsed profile data APIs were not tested in our benchmark.

Which APIs failed on LinkedIn in the benchmark?

In the September 16, 2026 run, Firecrawl, Scrapingdog and Zyte returned LinkedIn on none of five attempts. All three ran on a setting below their current anti-bot mode. Decodo returned 3 of 5 and ScrapingAnt 1 of 5.

Sources

• Web Data Frontier Benchmark results, String, September 16, 2026 run
• Web Data Frontier Benchmark harness, GitHub
• String fetch overview, String docs
• Access control and KYC, String docs
• Structured extraction, String docs
• String pricing, checked October 28, 2026
• LinkedIn's war against bot scrapers ramps up as AI gets smarter, Bloomberg Law, December 18, 2025
• LinkedIn User Agreement, effective November 3, 2025
• Someone else's LinkedIn scraper tool sat dead for four days, r/automation
• LinkedIn Corporation v. Nubela Pte. Ltd., N.D. Cal. docket