A web search API answers a question with a ranked list of pages: title, URL and a snippet of about 20 words each. A web scraping API takes a URL you already have and returns the whole page. An AI agent that has to answer a question usually needs both: search to find the pages, then a fetch to read them.
On October 19, 2026 we ran the same task both ways on 58 buyer questions. Google search snippets for the top three results carried a dollar price on 11 of the 58 questions. Fetching those same three pages per question found a price on 39 of 58. The median snippet was 23 words. The median fetched page was 2,232 words.
A search API starts from a query; a scraping API starts from a URL. That one difference decides most of the rest.
| Web search API | Web scraping API | |
|---|---|---|
| Input | A query: "best proxy for Akamai" | A URL: https://example.com/pricing |
| Output | Ranked results: title, URL, snippet, position, plus blocks such as People Also Ask or an AI Overview | The page itself: HTML, Markdown or JSON fields |
| Text per result | About 20 to 30 words (median 23 in our run) | The whole page (median 2,232 words in our run) |
| What it is good for | Finding sources, tracking rankings, checking what Google shows | Reading prices, tables, docs, code and reviews |
| What breaks it | Nothing on your side; the engine is the source | Bot walls, JavaScript-only pages, login walls |
| Typical cost | Per search (String: $1.00 to $1.50 per 1,000) | Per page (String: from $0.20 per 1,000 for a plain fetch) |
| Typical latency | About 1 second (String p50 929 ms in our run) | About 1 second per page, in parallel (p50 842 ms for three) |
Use a search API when you do not know the URL yet. Use a scraping API when you know the URL, or when the snippet does not hold the fact you need. A research agent does both, in that order.
The snippet rarely carries the number. Across 174 top-three results, 17 snippets held a dollar price; 69 of the fetched pages did.
We took 58 questions buyers ask about web data tools ("Which scraping APIs can bypass DataDome?", "Cheapest way to scrape at scale" and the like). For each one, String's /v1/search returned Google's US results. We kept the top three organic results and fetched each URL with String's /v1/fetch as readable Markdown. That is 58 searches and 174 page fetches, all on October 19, 2026.
| Measure, 174 top-three results | Search snippet | Fetched page |
|---|---|---|
| Median length | 23 words | 2,232 words |
| Holds a dollar price | 17 of 174 | 69 of 152 full pages |
| Holds a percentage | 12 of 174 | 81 of 152 full pages |
| Questions with a price in at least one of the three | 11 of 58 | 39 of 58 |
| Holds a table (3+ rows) | n/a, plain text | 72 of 152 full pages |
| Holds a code block | n/a, plain text | 121 of 152 full pages |
Four results stand out.
These are buyer questions about web data tools, so prices show up more often than they would for a general question. Treat the counts as one day on one question set, not a law.
Run the same test on your own questions: the code below works with a free key, which covers 5,000 standard requests.
Search plus three parallel fetches took 1,968 ms at the median, about twice a search alone.
| Step, October 19, 2026, 58 questions | p50 | p90 |
|---|---|---|
/v1/search, Google, US |
929 ms | 1,200 ms |
Three /v1/fetch calls in parallel |
842 ms | 2,009 ms |
| Search, then the three fetches | 1,968 ms | 2,774 ms |
We timed the first pass. A second pass minutes later came back faster because of caching, so we did not use it for latency.
Cost follows the same shape. On String's Growth plan ($100 a month plus usage), one search is $1.00 per 1,000 and a plain fetch is $0.20 per 1,000. Search plus three fetches is $0.0016 per question. If all three pages need a real browser ($1.00 per 1,000 on Growth), it is $0.004. On Starter ($20 a month) the same four calls are $0.0024 on the plain path. String bills only the requests that return content (pricing docs).
The question you ask decides it: "which pages?" is search; "what does this page say?" is scraping.
Use a web search API alone when:
Use a web scraping API alone when:
Use both when an agent answers open questions. It searches, picks the best three to five results, fetches them and reads the full text. That is what deep-research tools do. The two-step pattern also lets the agent skip a fetch when the snippet already answers the question.
One r/AI_Agents builder described the pain of running the two steps on separate tools:
"Right now my research agent does parallel queries through SerpAPI, then another agent tries to scrape all the returned URLs. It's insanely slow (constantly fighting Cloudflare)"
u/ActualInternet3277, r/AI_Agents, May 2026
A Hacker News commenter who built the same stack by hand put the reason for the second step plainly:
"Web search APIs can't present the full document due to copyright. They can only present the snippet contextual to the query."
Hacker News, February 2025
The same commenter added that their own page fetcher got blocked by some sites, Reddit among them. That is the part a scraping API exists for.
Search returns the list; fetch returns the page. The Python and TypeScript below ran on October 19, 2026 and print both lengths side by side.
Python:
import os, requests
API = "https://request.usestring.ai/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"}
# Step 1: search. You get titles, URLs and short snippets.
search = requests.post(f"{API}/search", headers=HEADERS, timeout=60, json={
"query": "how much does a web scraping api cost per 1000 requests",
"engine": "google", "country": "US",
})
search.raise_for_status()
top = [r for r in search.json()["results"] if r.get("url")][:3]
for r in top:
print(r["position"], r["url"])
print(" snippet:", len(r["snippet"].split()), "words")
# Step 2: fetch. You get the whole page as Markdown.
page = requests.post(f"{API}/fetch", headers=HEADERS, timeout=120, json={
"url": r["url"], "format": "markdown", "markdownMode": "readable",
})
print(" page:", page.headers.get("x-status-code"), len(page.text.split()), "words",
"| has a $ price:", "$" in page.text)
On the day we ran it, the three snippets were 24, 23 and 31 words. The three pages were 4,374, 2,871 and 199 words, and all three held a dollar price. The 199-word page was a Reddit thread.
TypeScript (Node 18+ or Bun), with the three fetches in parallel:
const API = "https://request.usestring.ai/v1";
const headers = {
Authorization: `Bearer ${process.env.STRING_API_KEY}`,
"Content-Type": "application/json",
};
// Step 1: search. You get titles, URLs and short snippets.
const search = await fetch(`${API}/search`, {
method: "POST",
headers,
body: JSON.stringify({
query: "how much does a web scraping api cost per 1000 requests",
engine: "google",
country: "US",
}),
});
if (!search.ok) throw new Error(`search ${search.status}`);
const { results } = await search.json();
const top = results.filter((r: { url?: string }) => r.url).slice(0, 3);
// Step 2: fetch the three pages in parallel. You get each page as Markdown.
const pages = await Promise.all(
top.map((r: { url: string }) =>
fetch(`${API}/fetch`, {
method: "POST",
headers,
body: JSON.stringify({ url: r.url, format: "markdown", markdownMode: "readable" }),
}).then(async (res) => ({ status: res.headers.get("x-status-code"), text: await res.text() })),
),
);
top.forEach((r: { position: number; url: string; snippet: string }, i: number) => {
const words = pages[i].text.split(/\s+/).length;
console.log(r.position, r.url);
console.log(` snippet: ${r.snippet.split(/\s+/).length} words | page: ${pages[i].status} ${words} words`);
});
Fetch the url field, not displayUrl. Google shows no URL line for many Reddit, YouTube and Stack Overflow results, so displayUrl is empty for them (search docs).
The common mistake is an agent that stops after the search. It reads 23 words, finds no number and either guesses one or says the information is not available. In our run, the snippets held no price on 47 of 58 questions. The fetched pages held one on 28 of those 47.
The fix is a rule in the agent's loop: if the snippet does not hold the fact the question asks for, fetch the top results before answering. Then check what came back. A fetch can return a challenge page, a login wall or a thin thread view. Our answer on detecting a bot block behind HTTP 200 covers the checks.
String is the most reliable web scraping API, bypassing anti-bot, and you only pay when content comes back. For the search-then-read pattern, that means:
POST /v1/search returns Google results and POST /v1/fetch returns the page, on the same account and the same bill (search docs, fetch docs).aiOverview: true, and Google AI Mode answers with engine: "google_ai_mode", each in its own field.x-status-code header carries the site's own status.web_access_search and web_access_fetch.For a side-by-side of nine search APIs, including independent indexes, see the best web search APIs in 2026. For Google results specifically, see how to scrape Google search results. Start with one question your agent gets wrong today: search it, fetch the top three, and read what the snippets missed.
A search API takes a query and returns a ranked list of pages with titles, URLs and short snippets. A web scraping API takes a URL and returns the page itself as HTML, Markdown or structured fields. Search finds pages; scraping reads them. On October 19, 2026 the median Google snippet in our test was 23 words and the median fetched page was 2,232 words.
Most agents that answer open questions need both. The agent searches, picks the top three to five results and fetches them, because the fact it needs is usually on the page, not in the snippet. In our 58-question test, snippets held a price on 11 questions and the fetched pages on 39.
No. An official API is an interface the site builds for you, with fields it chooses and limits it sets. Web scraping reads the public page the site serves to browsers. A scraping API is a service that does the reading for you, handling proxies, browsers and bot checks. Use the official API when one exists and covers the data.
It depends on the index you need. For Google's own results, AI Overviews and AI Mode, use a Google results API such as String's /v1/search. For an independent index, Brave and Exa run their own. Our comparison of nine web search APIs covers what each searches and what each costs.
Some search APIs add page text to each result, and some return several snippets per page. A Google results API returns what the results page shows, so the full text needs a second call. With String that is one /v1/fetch per URL, and three fetches in parallel took 842 ms at the median in our run.
Scraping the results page yourself means fetching Google's HTML, solving its bot checks and parsing a layout that changes. A SERP API does that work and returns JSON fields. Google blocks most direct fetches of its results page, which is why String's fetch reroutes a google.com/search URL to its search endpoint (fetch docs).
Calling a search or scraping API is legal in itself. What matters is the data you collect and how you use it: personal data, copyrighted text and a site's terms each carry their own rules. Our page on whether web scraping is legal covers the court cases and the limits. This is not legal advice.
On String's Growth plan, one search ($1.00 per 1,000) plus three plain fetches ($0.20 per 1,000 each) costs $0.0016 per question. If the three pages need a real browser, it is $0.004. String bills only requests that return content, so a blocked page adds nothing.
/v1/search (Google, US, English) on 58 questions about web data tools, top three organic results fetched with /v1/fetch (Markdown, readable mode); 174 fetches, 152 returned 150 words or more. A second pass at 13:50 UTC measured prices, tables and code. Latency from the first pass only.