How to scrape stock market data with Python in 2026

Last updated: October 12, 2026. Benchmark figures come from the September 16, 2026 run of the Web Data Frontier Benchmark. The script on this page ran on October 12, 2026, and the output shown is what it printed. We build String, the API the script calls.

To scrape stock market data with Python, read two Yahoo Finance URLs per ticker: the quote page for the current price and key stats, and the chart JSON endpoint for daily open, high, low, close and volume. On October 12, 2026, the script below returned quotes and 66 daily bars for AAPL, MSFT and NVDA in 4.1 seconds.

Getting Yahoo to answer is the hard part. A plain Python requests call got HTTP 429 "Too Many Requests" on its first request, before any volume. In our September 16, 2026 benchmark, 4 of 16 scraping APIs returned all six finance sites we test on every attempt.

This page is part of our series on the best web scraping APIs, tested.

On this page: TL;DR · Where String fits · What people run into · Why a plain request fails · Ways to get stock data · What 16 APIs did on six finance sites · Step 1 to Step 5 · How to avoid getting blocked · How String gets stock market data · The 429 that is not a rate limit · What it costs · FAQ

TL;DR

• Yahoo Finance answered Python's default requests client with HTTP 429 and a 23-byte body, "Edge: Too Many Requests", on the first request of the day on October 12, 2026.
• A Chrome User-Agent fixed the quote page (20 of 20 requests returned the price) but not the chart JSON endpoint, which stayed at 429 even with a Yahoo session cookie.
• curl_cffi with Chrome TLS impersonation, the library yfinance now depends on, read the chart JSON from the same home connection. It still got HTTP 401 from MarketWatch, WSJ and Barron's.
• In the September 16, 2026 benchmark, 4 of 16 scraping APIs returned all six finance sites on every attempt: String, Apify, Firecrawl and ScraperAPI.
• The six finance sites were easier than the average page: the field passed 78.1% of attempts on them, against 66.6% across all 100 sites.

Where String fits

String is a fit for one job here: getting the page or the JSON back when a finance site blocks your scraper.

• For: developers and small teams that pull quotes and daily history for a watchlist, build research or backtest datasets from public pages, or give an agent live quotes.
• Not for: tick data or trading signals. Quote pages are delayed; Bloomberg's AAPL page on October 12 carried "exchangeDelay":"15 Minutes". Use a broker or a licensed market-data feed for live trading. Not for pages behind a paywall or login either.
• Price: you pay only when content comes back. On the Starter plan ($20 a month), a plain fetch is $0.30 per 1,000 on standard proxies and $3.00 on premium; a browser fetch is $1.50 and $6.00. On Growth ($100 a month) the same four rates are $0.20, $2.00, $1.00 and $4.00. The first 5,000 standard requests are free (pricing).
• Integrations: a REST API you call from Python or any language, an MCP server for agents, and String Actors on Apify Store.
• Limits: 60 requests a second per account with a 3,600-request burst, and no cap on concurrency.
• Need a managed feed instead? String also builds web-data signals for investment teams, owned by an engineer through the reporting date (String for finance).

What people who scrape Yahoo Finance run into

The most common complaint is a 429 that arrives before any real volume. yfinance, the open-source Python library that reads Yahoo's endpoints, has a "Too Many Requests" issue with 141 comments (issue #2422). The person who opened it in April 2025 wrote that they had "tried using different IP addresses, so it's not my account hitting a rate limiter."

The first useful answer in that thread argues against paying anyone. A user found that Yahoo was checking the client, not counting requests, and that a free library fixed it:

"it works for me on a browser. It works with curl too. But somehow it just doesn't work with the Python requests module (with identical headers in all tests), consistently getting 429." TianqiMikeHu, yfinance issue #2422, April 30, 2025

They swapped in curl_cffi with Chrome impersonation, and it worked. Our test on October 12, 2026 agrees: from a home connection, curl_cffi read the chart JSON that plain requests could not.

The fix depends on where the code runs. A second user in the same thread hit the error only in the cloud:

"I got the same issue for my app in AWS Lambda, tried to redeploy and still rate limit." kianso7414, yfinance issue #2422, April 30, 2025

They added that the script worked when tested locally. Our page on why a scraper works locally but is blocked in Docker or Lambda covers that split.

The block also moves. In May 2025 a user posted a plain curl to the chart endpoint that came back HTTP/2 429 with content-length: 23 and the body "Edge: Too Many Requests" (DabeDotCom, issue #2422). That is byte for byte what Python's default client got from us on October 12, 2026.

The library fix has its own cost. In February 2026 a user asked yfinance for a fallback to normal curl, because "curl_cffi is pretty painful to build" on their system (barracuda156, issue #2692).

Why a plain request to finance sites fails

We sent the same URLs from a US home connection on October 12, 2026, with plain requests (default and Chrome User-Agent), with curl_cffi impersonating Chrome, and through String. Every row is one request unless noted.

"Price present" means the body contained 336.64, AAPL's close on Friday, October 9, 2026. The MarketWatch and WSJ 401 responses carried an x-datadome: protected header and a captcha-delivery script. The two Yahoo 429s are the interesting rows. They arrived on the first request, so backing off would not have fixed them. The section on the 429 that is not a rate limit, below, explains what changed them.

TradingView answered every client. If TradingView has the numbers you need, start there with plain requests.

Ways to get stock market data, compared

If a licensed feed fits your budget and use, use it. Scraping public quote pages suits watchlists, research and backtests, where a 15-minute delay does not matter.

What 16 scraping APIs did on six finance sites

The Web Data Frontier Benchmark sends the same 100 URLs to 16 scraping APIs, five attempts each. Six of the 100 are stock and market sites. The tested pages are the Yahoo Finance AAPL quote, the MarketWatch quote for PSKY, the TradingView NASDAQ-AAPL symbol page, a Barron's article, a WSJ live-coverage page and Bloomberg's markets page. The harness and targets are open source.

Successful loads out of five per site, read from the per-target cells of the September 16, 2026 run. A load passes only when the API returns a 2xx status and the body contains text from the real page, so a challenge page served at 200 counts as a failure. The raw result file is public.

* These zeros measure the vendor's API, not the site. ZenRows' API closed the connection without a response on every attempt from suite target 59 to the end, and Yahoo Finance (target 70) and TradingView (target 82) sit inside that stretch. Decodo's API returned HTTP 429 on every attempt from target 72, which covers TradingView. ZenRows' zero on Bloomberg (target 40) is outside the stretch and stands.

Two more caveats. Not every API ran in its strongest anti-bot mode: Scrapfly, Firecrawl, ScraperAPI, Zyte, ScrapingAnt, ScrapingBee, Scrapingdog and Nimble ran below the setting they offer today, so their cells may understate them. And the WSJ, Barron's and Bloomberg cells test an article, a live-coverage page and a markets page, not a quote page.

Four of 16 APIs returned all six sites on every attempt: String, Apify, Firecrawl and ScraperAPI. Nine had at least one site at zero. Yahoo Finance returned nothing to Bright Data, ScrapingAnt, ScrapingBee and Scrapingdog on any attempt, plus the ZenRows zero above. The field passed 375 of 480 loads, 78.1%, against 66.6% for the average page in the run. Finance is a middle-difficulty vertical, and the failures are concentrated in a few vendor and site pairs.

Step 1: pick the two Yahoo URLs

Yahoo Finance gives you each ticker twice: as an HTML quote page and as a JSON chart endpoint.

https://finance.yahoo.com/quote/AAPL/
https://query1.finance.yahoo.com/v8/finance/chart/AAPL?range=1mo&interval=1d

The quote page carries the price and the stats table: previous close, open, volume, market cap and P/E. The chart endpoint carries the time series. It is the same endpoint yfinance reads. range accepts values from 1d to max, and interval sets the bar size; Yahoo lists the valid ranges in the response under validRanges.

Step 2: fetch through one function

The fetch is one POST to String's fetch endpoint with the target URL.

import os
import requests

API = "https://request.usestring.ai/v1/fetch"

def fetch(url: str):
    response = requests.post(
        API,
        headers={"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"},
        json={"url": url, "countryCode": "US"},
        timeout=90,
    )
    response.raise_for_status()
    page = response.json()
    if page["statusCode"] != 200:
        raise RuntimeError(f"Yahoo returned {page['statusCode']} for {url}")
    return page["data"]

statusCode is Yahoo's own status, separate from the API's. A real 404 for a delisted ticker stays a 404. countryCode: "US" keeps the request on a US exit, so prices come back in the US market view.

Run this with a free key: export STRING_API_KEY=... and the first 5,000 standard requests cost nothing.

Step 3: parse the quote page

The price sits in a span with data-testid="qsp-price". The stats sit in fin-streamer elements, each named by a data-field attribute.

This is the markup from the October 12 run, trimmed:

<span class="price yf-1n64cj base" data-testid="qsp-price">336.64 </span>
<fin-streamer data-value="340.42" data-field="regularMarketPreviousClose">340.42</fin-streamer>
<fin-streamer data-value="4.913T" data-field="marketCap">4.913T</fin-streamer>

Test IDs and data-field names change less often than Yahoo's class names, which carry a generated suffix (yf-1n64cj). Two regular expressions read them:

import re

FIELD = re.compile(r'<fin-streamer[^>]*data-field="(\w+)"[^>]*>([^<]+)<')
PRICE = re.compile(r'data-testid="qsp-price">([^<]+)<')
NAME = re.compile(r"<title>(.+?) \(")

def quote(ticker: str) -> dict:
    html = fetch(f"https://finance.yahoo.com/quote/{ticker}/")
    price = PRICE.search(html)
    if not price:
        # A 200 without the price block is a consent page, a block page or a redesign.
        raise RuntimeError(f"HTTP 200 but no price on the {ticker} quote page")
    stats = {}
    for field, value in FIELD.findall(html):
        stats.setdefault(field, value.strip())  # first match is the quote's own stats table
    return {
        "ticker": ticker,
        "name": NAME.search(html).group(1),
        "price": float(price.group(1).replace(",", "")),
        "previous_close": stats.get("regularMarketPreviousClose"),
        "open": stats.get("regularMarketOpen"),
        "volume": stats.get("regularMarketVolume"),
        "market_cap": stats.get("marketCap"),
        "pe": stats.get("trailingPE"),
    }

The page also streams index futures (ES=F, NQ=F) in fin-streamer tags with a regularMarketPrice field, so do not read the price from the first regularMarketPrice you find. The qsp-price test ID belongs to the ticker you asked for.

The empty-result check matters most. A consent page, a block page and a redesign all arrive as HTTP 200. Our page on detecting a block page that returns HTTP 200 covers that failure in depth.

Step 4: pull price history from the chart JSON

The chart endpoint returns one array per field, aligned by index with a timestamp array.

import json
from datetime import datetime, timezone

def history(ticker: str, period: str = "1mo", interval: str = "1d") -> list:
    url = f"https://query1.finance.yahoo.com/v8/finance/chart/{ticker}?range={period}&interval={interval}"
    body = fetch(url)
    data = json.loads(body) if isinstance(body, str) else body  # JSON pages can come back already parsed
    result = data["chart"]["result"][0]
    q = result["indicators"]["quote"][0]
    rows = []
    for i, ts in enumerate(result["timestamp"]):
        if q["close"][i] is None:  # Yahoo leaves gaps as null on halted or partial bars
            continue
        rows.append({
            "ticker": ticker,
            "date": datetime.fromtimestamp(ts, timezone.utc).strftime("%Y-%m-%d"),
            "open": round(q["open"][i], 2),
            "high": round(q["high"][i], 2),
            "low": round(q["low"][i], 2),
            "close": round(q["close"][i], 2),
            "volume": q["volume"][i],
        })
    return rows

Each timestamp is the bar's open, 9:30 a.m. New York time for a daily bar, so the UTC date is the trading date. The meta block also carries currentTradingPeriod with the session's start and end. On October 12, 2026 it read 9:30 a.m. to 4:00 p.m. EDT, with pre-market from 4:00 a.m.

Step 5: loop the tickers and save a CSV

Prices mean something as a series, so the script appends every run to one CSV, the same pattern as our flight price script.

import csv
import sys
import time

if __name__ == "__main__":
    args = sys.argv[1:]
    out = None
    if "--csv" in args:
        i = args.index("--csv")
        out = args[i + 1]
        args = args[:i] + args[i + 2:]
    tickers = args or ["AAPL"]
    start = time.time()
    all_rows = []
    for t in tickers:
        q = quote(t)
        print(f"{t:<5} {q['name'][:24]:<24} ${q['price']:>8.2f}  open {q['open']}  prev {q['previous_close']}  "
              f"vol {q['volume']}  cap {q['market_cap']}  P/E {q['pe']}")
        rows = history(t)
        all_rows += rows
        last = rows[-1]
        print(f"      {len(rows)} daily bars {rows[0]['date']} to {last['date']}, last close ${last['close']:.2f}")
    if out:
        new = not os.path.exists(out)
        with open(out, "a", newline="") as f:
            w = csv.DictWriter(f, fieldnames=list(all_rows[0]))
            if new:
                w.writeheader()
            w.writerows(all_rows)
        print(f"appended {len(all_rows)} rows to {out}")
    print(f"done in {time.time() - start:.1f} s")

Run it with pip install requests, export STRING_API_KEY=..., then:

python stock_prices.py AAPL MSFT NVDA --csv prices.csv

This is what it printed on October 12, 2026, before the US market opened, so the price is Friday's close:

AAPL  Apple Inc.               $  336.64  open 331.79  prev 340.42  vol 37,881,199  cap 4.913T  P/E 38.61
      22 daily bars 2026-09-10 to 2026-10-09, last close $336.64
MSFT  Microsoft Corporation    $  535.07  open 528.23  prev 522.61  vol 18,500,612  cap 3.973T  P/E 29.81
      22 daily bars 2026-09-10 to 2026-10-09, last close $535.07
NVDA  NVIDIA Corporation       $  229.28  open 233.88  prev 230.48  vol 84,645,559  cap 5.536T  P/E 28.99
      22 daily bars 2026-09-10 to 2026-10-09, last close $229.28
appended 66 rows to prices.csv
done in 4.1 s

The CSV loads straight into pandas with pd.read_csv("prices.csv", parse_dates=["date"]). Schedule the script once a day after 4:00 p.m. New York time, and the file becomes a price history that you own.

How to avoid getting blocked on finance sites

Finance sites block in three different ways, and each needs a different fix.

• Yahoo checks the client. The 429 arrived on request one. A Chrome User-Agent fixed the quote page; Chrome TLS (curl_cffi) fixed the chart JSON from a home IP. Backoff alone did not change it.
• MarketWatch, WSJ and Barron's run DataDome. All three returned 401 to every local client we tried, curl_cffi included. The benchmark labels all three DataDome.
• Bloomberg runs PerimeterX. Plain requests got "Are you a robot?"; curl_cffi got the page on one request. One request is not a crawl.
• Cloud IPs change the answer. The Lambda report above is the pattern: a script that passes at home fails on a server.
• Test the body, not the status. Check for the price on every response, as Step 3 does, and treat a missing price as a failure.
• Pace your requests. Two requests per ticker per day is enough for a daily series. Our page on handling rate limits covers backoff, and our page on fixing a 403 covers headers and TLS.

How String gets stock market data

String is the most reliable web scraping API, bypassing anti-bot, and you only pay when content comes back. For stock market pages, that means:

• One call per URL. You send POST /v1/fetch with the URL, and String picks a plain request, a premium residential proxy or a real browser (fetch docs).
• The billed path is in the response. On October 12, 2026, the x-billed-request-type header read request_standard for the Yahoo quote page, the Yahoo chart JSON, MarketWatch, Bloomberg and Barron's, and request_premium for WSJ.
• JSON endpoints come back as JSON. The Yahoo five-day chart endpoint returned the same JSON yfinance reads, with a 200, in 0.45 seconds.
• String returned all six finance sites on every attempt. In the September 16, 2026 run, String was one of four of 16 APIs to return Yahoo Finance, MarketWatch, WSJ, Barron's, Bloomberg and TradingView five times out of five.
• A failed fetch is not billed. String charges only when content comes back (pricing).
• A daily watchlist costs cents. Fifty tickers at two requests each on 22 trading days is 2,200 requests a month: $0.66 at the Starter plan's standard fetch rate, and inside the free 5,000.

The 429 that is not a rate limit

Yahoo's 429 on October 12, 2026 was a client check with a rate-limit label.

Python's default client got "Edge: Too Many Requests" on its first request of the day. A Chrome User-Agent turned the quote page into 20 straight 200s at about one request a second. The chart JSON then answered 429 to requests from the same connection, with and without a cookie from finance.yahoo.com. curl_cffi impersonating Chrome got 200 from both query1 and query2 less than a minute later.

So a backoff loop on this 429 waits for something that will not change. Change the client, or route the request: curl -X POST https://request.usestring.ai/v1/fetch -H "Authorization: Bearer $STRING_API_KEY" -d '{"url":"https://query1.finance.yahoo.com/v8/finance/chart/AAPL?range=1mo&interval=1d"}'.

One caveat on this finding. It is one US home connection on one day. Yahoo's check has moved before, as the yfinance thread shows, and it may score a cloud IP differently.

What it costs

Both Yahoo requests we logged through String on October 12, 2026 billed as request_standard: $0.30 per 1,000 on Starter and $0.20 on Growth. WSJ's quote page billed as request_premium: $3.00 per 1,000 on Starter and $2.00 on Growth.

The last row is where a licensed feed starts to make sense. The rates come from usestring.ai/pricing on October 12, 2026; you pay only for requests that return content.

Is it legal to scrape stock market data?

Reading public quote pages is legal in many places, but the data's owners set terms on its use.

The yfinance README says it "is not affiliated, endorsed, or vetted by Yahoo" and points users to Yahoo's terms for their rights to the data (yfinance, Yahoo terms). Exchange data is licensed, and republishing it is a different act from reading it for research. Keep to public pages, read each site's terms, and see whether web scraping is legal. This is not legal advice.

Start with the tickers you check by hand today. Run the script above on them with a free key, and compare the CSV a week from now.

FAQ

How do I scrape stock market data with Python?

Fetch the Yahoo Finance quote page for the price and key stats, and the chart JSON endpoint for daily history. Parse the quote page by its qsp-price test ID and fin-streamer data-field names, and read the chart JSON as aligned arrays. On October 12, 2026, that returned quotes and 66 daily bars for three tickers in 4.1 seconds.

Is yfinance still usable in 2026?

Yes, for personal research from a home connection, in our test. yfinance now depends on curl_cffi, and curl_cffi impersonating Chrome read Yahoo's chart JSON on October 12, 2026, where plain requests got a 429. Its README says Yahoo's API is for personal use only, and users still report failures from cloud hosts such as AWS Lambda.

Why does Yahoo Finance return 429 Too Many Requests?

In our test it was a client check, not a count. Python's default client got the 429 on its first request. A Chrome User-Agent fixed the quote page, and Chrome TLS impersonation fixed the chart JSON. If a 429 arrives before any volume, change the client or route the request instead of backing off.

Is there an API to get stock data?

Yes. Brokers and licensed market-data vendors sell quote and history APIs, and that is the supported route for trading. The SEC's data.sec.gov APIs are free and need no key, but they carry filings and financial statements, not prices. Yahoo has no official public API; yfinance reads its unofficial endpoints.

How can I scrape data from Yahoo Finance without getting blocked?

Use a browser User-Agent for quote pages, Chrome TLS impersonation for the chart JSON, two requests per ticker per day, and a check that the price is in every response. From a cloud server, route through a scraping API. In the September 16, 2026 benchmark, 10 of 16 APIs returned Yahoo Finance on all five attempts.

Can I scrape Bloomberg, WSJ or MarketWatch?

Their public pages came back through String on October 12, 2026, all with the price present. Locally, MarketWatch, WSJ and Barron's returned a DataDome 401 to every client we tried, and Bloomberg returned a PerimeterX page to plain requests. Articles behind their paywalls are out of scope for any scraper.

Is scraped stock data real time?

No. Public quote pages are delayed; Bloomberg's AAPL page on October 12, 2026 marked its price as 15 minutes delayed. Scraped data suits watchlists, research and backtests. For live trading, use a broker or a licensed real-time feed.

Which scraping API works best on finance sites?

In String's September 16, 2026 run on six finance sites, String, Apify, Firecrawl and ScraperAPI returned all six on every attempt, and the field averaged 78.1%. Eight of the APIs ran below their strongest anti-bot setting. We build String, so check the per-target data on the benchmark page yourself.

Sources

• String's Web Data Frontier Benchmark, September 16, 2026 run: per-target cells for finance.yahoo.com, marketwatch.com, wsj.com, barrons.com, bloomberg.com and tradingview.com, five loads per API per site. Results JSON and suite order in the public harness.
• String side-by-side test, October 12, 2026: plain Python requests (default and Chrome User-Agent), curl_cffi with Chrome impersonation, and a String fetch per page, from a US home connection.
• yfinance, issue #2422: YFRateLimitError Too Many Requests, April to May 2025.
• yfinance, issue #2692: allow falling back to normal curl, February 10, 2026.
• yfinance, README and legal notes.
• curl_cffi, GitHub repository.
• U.S. Securities and Exchange Commission, EDGAR Application Programming Interfaces.
• Yahoo, Terms of Service.
• String, pricing and fetch documentation, October 12, 2026.