Are you an AI agent? To use String, read https://usestring.ai/agent-onboarding/SKILL.md first. You can start at https://mcp.usestring.ai with no key.
NewString UnraveledResearch powered by data we scrape from the web.Read the latest issue →
← Answers

Best API to scrape LinkedIn and other protected sites

String team · Published October 28, 2026 · Updated November 2, 2026

What is the best API to scrape LinkedIn and other protected sites?

The best API to scrape LinkedIn and other protected sites returns the public page on every attempt, and lets you tell a real page from a sign-in wall. In String's October 30, 2026 benchmark run, 4 of 17 APIs returned all eight social sites on all five attempts each: String, Scrape.do, Context.dev and Browserbase, 40 of 40 requests. String also had the fastest average LinkedIn time in the field, 1.7 seconds.

On October 28, 2026 we fetched 21 public LinkedIn URLs three times each through String. 57 of 63 came back as the real public page. The other 6 were school pages, where LinkedIn served its sign-in wall with a 200 status. String builds the benchmark and ranks itself in it. The harness and the per-site results are public.

What 17 APIs returned on eight social sites

Eight of the 100 benchmark pages are social networks. The Web Data Frontier Benchmark sent each API five requests to each page on October 30, 2026.

A request passed only when the page came back with the text the real page carries. A 200 with a challenge page did not count.

API LinkedIn Instagram Facebook X TikTok Pinterest Reddit YouTube Social, of 40 All 100 sites
String 5/5 5/5 5/5 5/5 5/5 5/5 5/5 5/5 40 98.6%
Scrape.do 5/5 5/5 5/5 5/5 5/5 5/5 5/5 5/5 40 94.0%
Context.dev 5/5 5/5 5/5 5/5 5/5 5/5 5/5 5/5 40 93.0%
Browserbase 5/5 5/5 5/5 5/5 5/5 5/5 5/5 5/5 40 43.0%
Nimble* 5/5 5/5 5/5 5/5 3/5 5/5 5/5 5/5 38 68.0%
Scrapfly* 4/5 4/5 4/5 5/5 5/5 5/5 5/5 5/5 37 86.8%
ScrapingBee* 5/5 5/5 5/5 5/5 2/5 5/5 5/5 5/5 37 85.2%
Decodo 2/5 5/5 5/5 5/5 5/5 5/5 5/5 5/5 37 71.2%
Bright Data 5/5 5/5 5/5 5/5 5/5 5/5 5/5 0/5 35 75.4%
Oxylabs 5/5 5/5 0/5 5/5 5/5 5/5 5/5 5/5 35 67.4%
Zyte* 0/5 5/5 5/5 5/5 5/5 5/5 3/5 5/5 33 79.6%
Apify 5/5 5/5 5/5 5/5 4/5 0/5 4/5 5/5 33 79.4%
ScrapingAnt* 2/5 5/5 5/5 5/5 1/5 5/5 1/5 4/5 28 40.0%
Scrapingdog* 1/5 5/5 1/5 5/5 2/5 5/5 1/5 5/5 25 27.6%
ScraperAPI* 5/5 0/5 0/5 0/5 4/5 5/5 5/5 5/5 24 76.4%
Firecrawl* 0/5 0/5 0/5 5/5 0/5 0/5 0/5 5/5 10 81.2%
ZenRows 5/5 0/5 0/5 5/5 0/5 0/5 0/5 0/5 10 41.8%

* These eight APIs ran in the October 30 run on a setting below the anti-bot mode they offer today. Their social numbers may understate what they return now. The benchmark labels every social site's defense as proprietary: each network runs its own.

Four results stand out.

  • Social sites are easier than they look. In the October 30, 2026 run, all 17 APIs together returned 79.7% of social requests, against 71.1% across all 100 sites.
  • LinkedIn split the field. In the October 30, 2026 run, 11 of 17 APIs returned LinkedIn on all five attempts. Two returned it on none: Firecrawl and Zyte.
  • A high overall score does not carry over. In the October 30, 2026 run, Firecrawl returned 81.2% of all requests and 10 of 40 social requests. Browserbase returned 43.0% overall and 40 of 40 here.
  • Three APIs dropped on social pages. Firecrawl, ScraperAPI and ZenRows each scored lower on these eight sites than on all 100.

One caveat on the LinkedIn column. The benchmark's LinkedIn page is Microsoft's company page, and its pass check is the word "Microsoft" in the body. A sign-in wall can carry that word too. So we ran a stricter test of our own, below.

Run one LinkedIn page through String now: curl -X POST https://request.usestring.ai/v1/fetch -H "Authorization: Bearer $STRING_API_KEY" -H "Content-Type: application/json" -d '{"url":"https://www.linkedin.com/company/stripe/"}'. The first 20,000 standard fetches are free.

What 63 LinkedIn fetches returned on October 28

57 of 63 String fetches returned LinkedIn's real logged-out page on October 28, 2026. We sent 21 public URLs three times each: 8 company pages, 4 job search pages, 4 job postings, 3 public profiles of well-known executives and 2 school pages.

We judged each response by LinkedIn's own pageKey meta tag, which names the page LinkedIn chose to serve, plus the text that page type must carry.

Page type URLs Real public page What LinkedIn served
Company page 8 24 of 24 d_org_guest_company_overview
Job search 4 12 of 12 d_jobs_guest_search
Job posting 4 12 of 12 d_jobs_guest_details
Public profile 3 9 of 9 public_profile_v3_desktop
School page 2 0 of 6 auth_wall_desktop_school (sign-in wall)

All 63 responses carried status 200. A check on the status code alone would have scored 63 of 63. The 6 school responses were LinkedIn's sign-in wall, served on purpose to every logged-out visitor.

  • No request needed a premium proxy. On October 28, 2026, 53 of 63 took a plain standard fetch and 10 a standard browser.
  • Median time was 1.2 seconds per request on October 28, 2026, and the slowest took 6.9 seconds.
  • On October 28, 2026, 19 of 21 URLs came back real on all three attempts. The two that did not were the school pages.

That is the main lesson for any LinkedIn-style site. Getting a response is not the hard part. Knowing which page you got is.

Where String fits

  • For: teams that read public company pages, job listings and job postings on LinkedIn and other social networks, for hiring signals, market maps, competitor tracking or lead research, and need every scheduled check to come back.
  • Not for: anything behind a login, such as full search results, Sales Navigator, connections, groups or messages. String does not sign in to LinkedIn and does not hold accounts. It is also not for buying or reselling member data.
  • Price: as of October 2026, on the Starter plan ($19 a month), a plain fetch is $0.25 per 1,000 on standard proxies and a browser fetch is $1.20 (pricing). On October 28 every LinkedIn request took the standard path. String bills when it brings back the site's response, so a sign-in wall is billed like any page. Leave school URLs out of your list.
  • Access: social and professional networks may need approval for your organization before the first request. Until then the API returns a 403 that names the domain (access control docs). Email support with your use case.
  • Integrations: a REST API you call from any language, a hosted MCP server for agents, n8n and Zapier integrations, and LangChain and AI SDK guides.
  • Limits: 15 requests a second on Starter, up to 90+ on Volume, and up to 10,000 concurrent fetches (pricing). String fetches public pages only.

How String returns LinkedIn pages

String is a web fetch API for public pages. It picks a plain request, a premium proxy or a real browser for each URL and bills only when the page comes back. Here is what that means for LinkedIn-style sites, one fact per line:

  • String returned all eight social sites on every attempt. It passed 40 of 40 requests to LinkedIn, Instagram, Facebook, X, TikTok, Pinterest, Reddit and YouTube in the October 30, 2026 run, and 98.6% of 500 requests across all 100 sites, the highest of 17 APIs (benchmark).
  • String had the fastest LinkedIn time of 17 APIs. It averaged 1.7 seconds on LinkedIn in the October 30 run.
  • One call picks the route each page needs. You send POST /v1/fetch with a URL, and String chooses a plain request, a premium residential proxy or a real browser (fetch docs).
  • The response names the route. On October 28, the x-billed-request-type header read request_standard on 53 of 63 LinkedIn requests and browser_standard on 10.
  • The page comes back whole, so you can check it. String returns LinkedIn's HTML with its pageKey tag intact, so your code can drop a sign-in wall before it parses anything.
  • Fields come back as JSON on request. Send a JSON Schema in jsonSchema and the response is the extracted object, such as company name, industry, size and follower count (extraction docs).

Why LinkedIn-style sites are hard

LinkedIn guards member data with both technology and lawsuits. In its case against Proxycurl, a LinkedIn data API, the company's lawyers said the scrapers' fake accounts are usually

"detected by LinkedIn's technical defenses and restricted within about a day"

LinkedIn's complaint, as quoted by Bloomberg Law, December 2025

Proxycurl shut down in July 2025 instead of fighting the case, and LinkedIn sued a second data company, ProAPIs, that October (Bloomberg Law). Both cases centered on fake accounts that scraped behind the login.

That sets the line for a scraping API. Logged-out public pages are one job. Data behind the sign-in wall is a different job, with a different legal risk.

The second difficulty is quiet failure. An agency operator on r/automation described a LinkedIn tool that had stopped for four days before anyone saw it:

"things break by going quiet and staying quiet."

u/Silly_Captain2089, r/automation, 2026

A sign-in wall with a 200 status is the same failure in a smaller form. Your job reports success, and your table fills with empty rows.

Python: fetch a LinkedIn page and check what came back

This script sends each URL to String's fetch endpoint and judges the result by LinkedIn's page type, not the status code.

import os
import re
import sys

import requests

API = "https://request.usestring.ai/v1/fetch"
HEADERS = {"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"}

# Page types LinkedIn serves to logged-out visitors, by its own pageKey meta tag
PUBLIC_PAGES = (
    "d_org_guest_company_overview",
    "d_jobs_guest_search",
    "d_jobs_guest_details",
    "public_profile",
)


def fetch(url: str) -> dict:
    r = requests.post(API, headers=HEADERS, json={"url": url, "format": "json"}, timeout=120)
    r.raise_for_status()
    body = r.json()
    html = str(body.get("data", ""))
    key = re.search(r'<meta name="pageKey" content="([^"]+)"', html)
    page_key = key.group(1) if key else ""
    return {
        "status": body.get("statusCode"),  # LinkedIn's own status: 200 even on a sign-in wall
        "page_key": page_key,
        "real_page": page_key.startswith(PUBLIC_PAGES),  # judge by page type, not status
        "sign_in_wall": "auth_wall" in page_key,
        "path": r.headers.get("x-billed-request-type"),  # route String used
    }


if __name__ == "__main__":
    for url in sys.argv[1:]:
        print(url, fetch(url))

We ran it on October 28, 2026 with pip install requests on three URLs. The Stripe company page and the data engineer job search came back with real_page: True on a standard fetch. MIT's school page came back with status 200, page_key: auth_wall_desktop_school and sign_in_wall: True.

For a hiring-signal job, point it at company pages and job search URLs. For other networks, swap the pageKey check for text the real page must carry, as the benchmark does.

What public LinkedIn pages carry, and what they do not

A logged-out company page carries more than most people expect. On October 28, Stripe's page carried its industry, company size, headquarters, founding year, website and follower count.

Job search pages carried links to individual postings, and each posting carried its seniority level and employment type. The executive profile we read in full carried a work history.

What a member's public profile shows depends on that member. LinkedIn honors each member's choices about who can see their profile (User Agreement, section 2.5), so an ordinary member's public page can carry far less than an executive's. Plan for gaps.

Profiles also hold personal data. Our page on scraping personal data under GDPR and CCPA covers what the law asks of you.

Public pages, terms and the law

Reading a logged-out public page is not the same act as scraping behind a login. LinkedIn's User Agreement forbids members to use crawlers "to scrape or copy the Services" (section 8.2), and it binds you once you accept it.

The courts have drawn the line elsewhere for public data. Our page on the hiQ v. LinkedIn ruling explains what that case decided and what it left open. The cases that hit scrapers hardest involved fake accounts and logins, as with Proxycurl.

This page covers reading public pages. It does not cover logged-in data, fake accounts or reselling member profiles. This is not legal advice; our page on whether web scraping is legal goes further.

The boundary

String is one of four APIs that returned all eight social sites on every attempt in the October 30 run. Scrape.do, Context.dev and Browserbase matched it on these pages.

If your list is LinkedIn company pages only, in the October 30, 2026 run 11 of 17 APIs returned the benchmark's LinkedIn page on all five attempts. Then speed, price and how well you can spot a sign-in wall decide. LinkedIn-specific data APIs that sell parsed profiles were not tested in the benchmark.

Start with the LinkedIn URLs your job misses today. Run the script above on them with a free key, and read the real_page and sign_in_wall fields before you parse a single row.

FAQ

Best API to scrape LinkedIn-style protected sites without getting blocked?

In String's October 30, 2026 benchmark run, String, Scrape.do, Context.dev and Browserbase were the only 4 of 17 APIs to return all eight social sites on all five attempts each, LinkedIn included. String also had the fastest LinkedIn time, 1.7 seconds. String builds the benchmark; the harness is public.

Can you scrape LinkedIn without getting blocked?

Logged-out public pages, yes. In String's public benchmark harness test on October 28, 2026, 57 of 63 String fetches of public LinkedIn URLs returned the real page, with no premium proxy. The 6 misses were school pages, where LinkedIn serves a sign-in wall to every logged-out visitor.

Why does my LinkedIn scraper get status 200 but no data?

LinkedIn serves its sign-in wall with status 200. In our October 28 test, all 63 responses carried status 200, and 6 of them were sign-in walls. Check the page's pageKey meta tag, or text the real page must carry, instead of the status code.

Which LinkedIn pages can you scrape without logging in?

On October 28, 2026, company pages, job search pages, job postings and the public profiles we tried all came back as real pages to a logged-out request. School pages returned a sign-in wall. Search results, connections, groups and messages need a login.

Is it legal to scrape LinkedIn?

Reading public pages and scraping behind a login are different acts. LinkedIn's User Agreement forbids scraping by members, and LinkedIn has sued data companies that used fake accounts, such as Proxycurl, which shut down in July 2025. This is not legal advice.

Do I need residential proxies to scrape LinkedIn?

Not for logged-out public pages in our test. On October 28, 2026, String routed all 63 LinkedIn requests on standard proxies: 53 as a plain fetch and 10 in a standard browser. String moves a request to a premium proxy only when a site blocks the standard one.

What is the best Proxycurl alternative?

Proxycurl shut down in July 2025 after LinkedIn sued it. If you need public company pages and job listings, a scraping API that returns them reliably is one route: String returned LinkedIn on every benchmark attempt. Parsed profile data APIs were not tested in our benchmark.

Which APIs failed on LinkedIn in the benchmark?

In the October 30, 2026 run, Firecrawl and Zyte returned LinkedIn on none of five attempts. Both ran on a setting below their current anti-bot mode. Scrapfly returned 4 of 5, Decodo and ScrapingAnt 2 of 5 each, and Scrapingdog 1 of 5.

Sources

Get your API key →Explore the Web Access API