How to keep an LLM's knowledge current with live web data

Last updated: October 23, 2026. Every number on this page comes from searches and page fetches we made on October 23, 2026 through String's search and fetch endpoints, from one US connection. Model training cutoffs come from each vendor's own docs, read the same day. The Python code ran the same day, and the output shown is what it printed. We build String, the API the code calls.

To keep an LLM's knowledge current with live web data, retrieve at question time: search the web, fetch the top pages, read each page's own date, and give the model today's date with the dated text. Tell it to prefer the newest source and to say when a source has no date. For sources you use again and again, also keep a RAG index fresh: re-fetch them on a schedule and re-embed only the pages whose text changed.

The model cannot do this from memory. On October 23, 2026 we searched 78 questions and dated the top 3 Google results for each. 71 of 233 pages (30%) were dated after June 2026, the newest training cutoff any of the three big model vendors publishes. 114 (49%) carried no date in their metadata, so a pipeline that does not look for one cannot tell the model how old the page is.

This page covers freshness. To pick a search API for an agent, see search API for AI agents. For when a snippet is enough and when to fetch the page, see web search API vs web scraping API. To compare search APIs side by side, see the best web search APIs.

On this page: TL;DR · Where String fits · What we measured · The named failure: the undated source · Step 1 to Step 6 · How to avoid getting blocked · How String keeps an LLM's sources current · What it costs · FAQ · Sources

TL;DR

• The training cutoff is a hard edge. Anthropic lists June 2026 for its current models, OpenAI April 30, 2026 for GPT-6 Astra, and Google March 2026 for Gemini 3.8 Flash.
• 30% of top results are newer than every one of those cutoffs. 71 of 233 top-3 Google results were dated after June 2026. On 54 of 78 questions, at least one of the top 3 was.
• Half the pages do not say when they were written. 114 of 233 had no date in JSON-LD or meta tags. On 20 time-sensitive questions (prices, rates, versions, news), 42 of 60 had none.
• Retrieve at question time for anything that moves. Search, fetch, read the date, and pass dated sources with today's date.
• Ask Google for recent results when the question is about now. A past-month filter raised the share of top pages dated after June 2026 from 15 of 60 to 26 of 59 on the time-sensitive questions.
• Keep an index fresh for the sources you trust. Re-fetch on a schedule, hash the text, and re-embed only what changed.

Where String fits

String fits two jobs on this page: the search and fetch calls at question time, and the re-fetch in a scheduled index refresh.

• For: teams building RAG pipelines, agents and assistants whose answers depend on facts that change: prices, versions, docs, rates and news. One API key gives Google results and the pages behind them.
• Not for: knowledge that changes less often than you upgrade your model. Your own databases, where change data capture from the source is the better feed. Pages behind your own login.
• Price: you pay only when content comes back, and the first 5,000 standard requests are free. Starter is $20 a month: search $1.50 per 1,000, fetch $0.30 per 1,000 on standard proxies. Growth is $100 a month: search $1.00 and fetch $0.20 per 1,000 (pricing, billing).
• Integrations: a REST API you call from Python, a hosted MCP server with web_access_search and web_access_fetch tools, a LangChain package and a Vercel AI SDK guide.
• Limits: 60 requests a second with a burst allowance of 3,600, and no cap on concurrent requests. The dateRange filter works on Google only. String returns pages; you bring the model, the vector store and the scheduler.

What we measured: how fresh are the pages a model retrieves

We dated 233 pages, the top 3 Google results for 78 questions, on October 23, 2026. For each question, String's /v1/search returned Google's US results. We fetched the top 3 organic URLs with /v1/fetch and read the dates each page declares about itself.

Two question sets. 58 are questions people ask about web data tools, a market where vendors and prices change every month. 20 are plainly time-sensitive: "bitcoin price today", "current federal funds rate", "kubernetes latest release", "world series 2026", "anthropic claude api pricing" and the like.

The rule for a page's date: the latest date in its JSON-LD (dateModified, datePublished, uploadDate, dateCreated) or its date meta tags (article:modified_time, article:published_time, og:updated_time and a few others). On a review or forum page that can be the newest review or post. "No date signal at all" means no such date, no <time datetime> tag and no Last-Modified header older than a day.

The training cutoffs we compared against, from each vendor's own docs on October 23, 2026:

A stated cutoff is an upper bound. Google's model card says users may find the model's knowledge "is limited to January 2025" in some domains.

We checked the extraction by hand on 28 pages. On 12 of 14 dated pages the metadata date matched the page, or the page showed no other date. One page showed a newer "updated" date than its metadata, and one carried a newer metadata date than anything it showed. Of 14 undated pages, 3 showed their own date only in the text, which a model can read only if your pipeline keeps it.

Caveats: one day, one US connection, one search engine. A page dated after June 2026 is newer than the model; that does not mean every fact on it is new. 12 dated pages carried a date from the two days before our fetch. If every one of those were a server stamp rather than an edit, the 71 would be 59 (25%).

The named failure: the undated source

The named failure is a confident answer built on a source with no date. It takes two forms.

1. The model answers from memory. The question is about something after its cutoff, retrieval is off or returns nothing useful, and the model fills the gap with what it learned in training.
2. The model reads a page and cannot tell how old it is. 49% of the top results we fetched carried no date in their metadata. The model sees the text, treats it as current, and states a two-year-old version number as today's.

Developers see the first form after every model launch. One Hacker News commenter tested a new Gemini model in July 2026:

"despite that the knowledge cut off is march 2026 it still knows nothing about 2025!" npn, Hacker News, July 21, 2026

Others in that thread disputed the test, so read it as one user's result. A year earlier, another commenter put the cost to developers plainly:

"You'll still need to provide this model with a lot of context to use it with any tooling or libraries with breaking changes or new features from the past ~year" nikcub, Hacker News, March 25, 2025

Retrieval has its own critic. A commenter on a May 2026 thread argued that search flattens the model's own value:

"If the LLM uses a search tool there's barely a difference between a 30B model and Opus or GPT 5.5, because it just bases its reply on the stuff that came up. Which is generally SEO junk." deaux, Hacker News, May 20, 2026

The commenter is right that retrieval is only as good as the pages it brings back. That is the case for reading each page's date and preferring the newest dated source (Steps 3 and 4), and for a curated index of sources you trust (Step 5).

Step 1: search at question time

Search with the user's question as written, and keep the top results. The function below calls String's /v1/search for Google US results. We ran it on Python 3.12 with requests 2.34.2. It needs a String API key in STRING_API_KEY.

Save the code blocks of Steps 1, 3 and 4 in order in one file, fresh_context.py.

"""Give an LLM current web knowledge at question time: search, fetch, date each page, build a dated context."""
import html, os, re, sys
from datetime import datetime, timezone

import requests

API = "https://request.usestring.ai/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['STRING_API_KEY']}"}


def search(question: str, recent: str | None = None, k: int = 3) -> list[dict]:
    body = {"query": question, "engine": "google", "country": "US", "language": "en"}
    if recent:  # "day", "week", "month" or "year": Google's own recency filter
        body["dateRange"] = recent
    r = requests.post(f"{API}/search", json=body, headers=HEADERS, timeout=60)
    r.raise_for_status()
    return [x for x in r.json()["results"] if x.get("url")][:k]

Each result carries title, url, snippet and position. Fetch url, not the display URL (search docs).

Run a search with a free key to see what comes back:

curl https://request.usestring.ai/v1/search \
  -H "Authorization: Bearer $STRING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "kubernetes latest release", "dateRange": "month"}'

On October 23, 2026 it returned 10 results. The top 3 were endoflife.date, Oracle's Kubernetes versions page and Confluent's release notes.

Step 2: ask for recent results when the question is about now

Google's own date filter is one field. Set dateRange to "day", "week", "month" or "year" when the user asks about news, prices or a recent release. Add "sortBy": "date" to put the newest results first (search docs).

We ran the 20 time-sensitive questions again with "dateRange": "month":

The filter changes what comes back, not only how old it is. For "latest python release" the top result became a security release for Python 3.11, not the newest version. Use the filter for news and prices. For "what is the latest version", search without it and let the page dates decide.

Here is the same question with and without the filter, through the full script from Step 4:

python fresh_context.py "best llm for coding"
python fresh_context.py "best llm for coding" month

 no date on page  https://www.vellum.ai/best-llm-for-coding
      2025-11-21  https://www.reddit.com/r/vibecoding/comments/1p2xrvi/best_llm_for_coding/
      2026-10-20  https://benchlm.ai/coding

prompt: 12,142 characters, 3 sources

      2026-09-25  https://www.datacamp.com/blog/best-llm-for-coding
      2026-10-20  https://benchlm.ai/coding
      2026-09-26  https://www.reddit.com/r/LocalLLM/comments/1wr3sut/best_coding_agent_llm/

prompt: 12,647 characters, 3 sources

Without the filter, one of the top 3 had no date and one was a Reddit thread from November 2025. With it, all three were dated, and the oldest was from September 25, 2026. Google's order moved between our first run at 10:10 UTC and this one at 15:16 UTC, so your results will differ.

Step 3: fetch the pages and read their dates

Fetch each URL with "format": "json". The response carries the site's own status code, its headers and the page HTML in data (response formats). Then read the date the page declares about itself.

def fetch(url: str) -> dict:
    r = requests.post(f"{API}/fetch", json={"url": url, "format": "json"}, headers=HEADERS, timeout=120)
    r.raise_for_status()
    return r.json()  # statusCode, headers, data (the HTML), finalUrl


DATE_KEYS = ("dateModified", "datePublished", "uploadDate")
META_KEYS = ("article:modified_time", "article:published_time", "og:updated_time")


def page_date(page_html: str) -> datetime | None:
    """The latest date the page declares in JSON-LD or date meta tags, else its first <time datetime>."""
    found = re.findall(r'"(?:%s)"\s*:\s*"([^"]+)"' % "|".join(DATE_KEYS), page_html)
    for tag in re.findall(r"<meta\b[^>]*>", page_html, flags=re.I):
        if any(f'"{k}"' in tag for k in META_KEYS):
            m = re.search(r'content="([^"]+)"', tag)
            found += [m.group(1)] if m else []
    dates = []
    for raw in found:
        try:
            d = datetime.fromisoformat(raw.strip().replace("Z", "+00:00"))
        except ValueError:
            continue
        d = d if d.tzinfo else d.replace(tzinfo=timezone.utc)
        if d <= datetime.now(timezone.utc):
            dates.append(d)
    if not dates:  # forums such as Reddit put the post time in a <time> tag instead
        first = re.search(r'<time\b[^>]*datetime="([^"]+)"', page_html)
        if first:
            try:
                dates.append(datetime.fromisoformat(first.group(1).replace("Z", "+00:00")))
            except ValueError:
                pass
    return max(dates, default=None)

Three rules come from the 233 pages we dated.

• Prefer metadata. 111 of the 119 dated pages carried the date in JSON-LD, the most common source. 72 carried a date meta tag.
• Fall back to <time> on forums. 40 of the 114 pages with no metadata date were Reddit threads. Every one carried the post time in a <time datetime> tag. Take the first one: later tags are replies.
• Do not trust Last-Modified alone. 61 pages sent the header, and on 44 it was within a day of the response's own Date header. That is the server's clock, not the content's age.

Drop dates later than now. A page that declares a future date has a bug, not news.

Step 4: give the model today's date and dated sources

Put each source in the prompt with its URL and its page date, and put today's date first. The model has no clock; without today's date it cannot judge "last month".

def visible_text(page_html: str, limit: int = 4000) -> str:
    body = re.sub(r"<(script|style|noscript|svg|template)\b.*?</\1\s*>", " ", page_html, flags=re.S | re.I)
    text = html.unescape(re.sub(r"<[^>]+>", " ", body))
    return re.sub(r"\s+", " ", text).strip()[:limit]


def build_context(question: str, recent: str | None = None) -> tuple[str, list[dict]]:
    sources = []
    for hit in search(question, recent):
        env = fetch(hit["url"])
        if not (200 <= env["statusCode"] < 300) or not isinstance(env["data"], str):
            continue  # a 404 or a non-HTML body: skip it rather than feed it to the model
        d = page_date(env["data"])
        sources.append({"url": env["finalUrl"], "title": hit["title"],
                        "date": d.date().isoformat() if d else "no date on page",
                        "text": visible_text(env["data"])})
    today = datetime.now(timezone.utc).date().isoformat()
    blocks = "\n\n".join(f"[{i}] {s['title']}\nURL: {s['url']}\nPage date: {s['date']}\n{s['text']}"
                         for i, s in enumerate(sources, 1))
    prompt = (f"Today is {today}. Answer from the sources below, not from memory.\n"
              f"Prefer the most recent dated source. Cite each claim as [n] with the source's page date.\n"
              f"If a source has no date, say so. If the sources disagree, say which is newer.\n\n"
              f"{blocks}\n\nQuestion: {question}")
    return prompt, sources


if __name__ == "__main__":
    question = sys.argv[1]
    recent = sys.argv[2] if len(sys.argv) > 2 else None
    prompt, sources = build_context(question, recent)
    for s in sources:
        print(f"{s['date']:>16}  {s['url']}")
    print(f"\nprompt: {len(prompt):,} characters, {len(sources)} sources")

Send prompt as the user message to your model's chat API. The instruction lines do the work: answer from the sources, prefer the newest dated one, cite the date, and flag a source with no date. A citation with a date lets the reader check how current the answer is.

Three settings to tune. k=3 keeps cost and context small; raise it for research questions. limit=4000 characters per page keeps the prompt near 12,000 characters. For cleaner text than a tag strip, fetch with "format": "markdown" and "markdownMode": "readable", which keeps JSON-LD as fenced code blocks; date meta tags do not survive the conversion (response formats).

Run Steps 1 to 4 with a free key: python fresh_context.py "your question" prints each source with its page date.

Step 5: keep an index fresh for the sources you use most

Retrieval at question time covers the open web. For the sources you return to (your vendor's docs, a changelog, a price page), keep them in a vector index and refresh it on a schedule. Re-fetch each source when it is due, hash its text, and re-embed only the pages that changed.

List the sources in sources.json next to the script, each with how often it should be re-fetched:

[
  {"url": "https://www.python.org/downloads/", "every_hours": 24},
  {"url": "https://endoflife.date/kubernetes", "every_hours": 24},
  {"url": "https://docs.langchain.com/oss/python/releases/changelog", "every_hours": 6}
]

Save this as refresh_index.py:

"""Keep a RAG index current: re-fetch each source when it is due, re-embed only the pages whose text changed."""
import hashlib, json, os, time
from datetime import datetime, timezone
from pathlib import Path

from fresh_context import fetch, page_date, visible_text

HERE = Path(__file__).resolve().parent
SOURCES = json.loads((HERE / "sources.json").read_text())  # [{"url": ..., "every_hours": 24}, ...]
STATE = HERE / "index_state.json"
FORCE = os.environ.get("FORCE") == "1"  # refetch everything now, ignoring every_hours


def embed_and_upsert(url: str, text: str, date: str) -> None:
    # Replace with your embedding model and vector store. Delete the page's old chunks first,
    # then insert the new ones with the page date as metadata, so retrieval can filter or sort by it.
    chunks = [text[i:i + 1500] for i in range(0, len(text), 1500)]
    print(f"  re-embedded {len(chunks)} chunks, page date {date}")


def run() -> None:
    state = json.loads(STATE.read_text()) if STATE.exists() else {}
    now = time.time()
    for src in SOURCES:
        url, saved = src["url"], state.get(src["url"], {})
        if not FORCE and now - saved.get("fetched_at", 0) < src["every_hours"] * 3600:
            print(f"NOT DUE    {url}")
            continue
        env = fetch(url)
        if not (200 <= env["statusCode"] < 300):
            print(f"FAILED     {url}  status {env['statusCode']}; keeping the indexed copy")
            continue
        text = visible_text(env["data"], limit=200_000)
        digest = hashlib.sha256(text.encode()).hexdigest()
        d = page_date(env["data"])
        date = d.date().isoformat() if d else "no date on page"
        if digest == saved.get("hash"):
            print(f"UNCHANGED  {url}")
        else:
            print(f"{'CHANGED' if saved else 'NEW':10} {url}")
            embed_and_upsert(url, text, date)
        state[url] = {"hash": digest, "page_date": date, "fetched_at": now,
                      "checked": datetime.now(timezone.utc).isoformat(timespec="seconds")}
    STATE.write_text(json.dumps(state, indent=2))


if __name__ == "__main__":
    run()

On October 23, 2026 we ran it three times: once from empty, once right after, and once with FORCE=1. The output, with the timestamp lines cut:

== run 1
NEW        https://www.python.org/downloads/
  re-embedded 15 chunks, page date no date on page
NEW        https://endoflife.date/kubernetes
  re-embedded 8 chunks, page date 2026-09-24
NEW        https://docs.langchain.com/oss/python/releases/changelog
  re-embedded 12 chunks, page date 2026-10-23
== run 2
NOT DUE    https://www.python.org/downloads/
NOT DUE    https://endoflife.date/kubernetes
NOT DUE    https://docs.langchain.com/oss/python/releases/changelog
== run 3 (FORCE=1)
UNCHANGED  https://www.python.org/downloads/
UNCHANGED  https://endoflife.date/kubernetes
UNCHANGED  https://docs.langchain.com/oss/python/releases/changelog

Four rules keep the index honest:

• Delete before you insert. A changed page's old chunks must go, or the index answers with both versions.
• Store the page date and the fetch time with every chunk. Retrieval can then filter out stale chunks or rank newer ones first, and the prompt can show both.
• Keep the last good copy when a fetch fails. A failed re-fetch is not a deleted page.
• Hash the text you embed, not the raw HTML. HTML carries session ids and ad markup that change on every fetch. Hash only the part you act on when a page has noise, such as a review count.

Run the script from cron, GitHub Actions or APScheduler. For locks, retries and exit codes on a schedule, see how to schedule recurring scraping jobs.

Step 6: choose question-time retrieval, a fresh index, or both

The age of the fact decides the method. Match each kind of knowledge to the cheapest method that keeps it current.

Fine-tuning is not on the list. It changes how a model writes, and it is a slow, costly way to add facts that will change again next week. A Hugging Face forum answer to "How can an LLM be trained on live data?" put it this way: "For general real-time LLM capabilities, however, I don't believe it's feasible with the current architecture and technology" (mahmutc, Hugging Face Forums, December 2024). The same answer points to RAG and agents instead.

Most production systems do both. The index answers the common questions fast. Question-time search catches what the index has never seen.

How to avoid getting blocked

Question-time retrieval fails quietly when a fetch is blocked: the model gets a challenge page or nothing, and answers from memory. The checklist:

• Check the destination status code and skip non-2xx pages, as Step 4 does. Do not pass a 403 page to the model.
• Check the body for a challenge page. A 200 can still be a wall; see how to detect a bot block that returns HTTP 200.
• Fetch the top results in parallel, but pace requests per site in a scheduled refresh.
• Re-fetch an index source no more often than it changes. A changelog every 6 hours; most docs once a day.
• Route the pages your own requests cannot get through an API built for blocked sites. For the techniques behind each step, see how to scrape without getting blocked.

In our run, 233 of 234 top-3 results came back through String. The one miss was a 404 at the destination.

How String keeps an LLM's sources current

String is the most reliable web scraping API, bypassing anti-bot, and you only pay when content comes back.

• String returned 233 of the 234 top-3 Google results we fetched on October 23, 2026. The one miss was a page that answered 404 at its own site.
• String's /v1/search takes Google's own date filter. dateRange limits results to the past hour, day, week, month or year, and "sortBy": "date" puts the newest first (search docs).
• "format": "json" returns the page HTML with the site's own status code and headers. One response carries everything page_date() reads: JSON-LD, meta tags, <time> tags and Last-Modified (response formats).
• String had the highest success rate of 16 scraping APIs in the September 16, 2026 benchmark run: 97.0% across 100 sites, five attempts per site (benchmark).
• A failed request is not billed. A blocked page in a refresh costs nothing (billing).
• The hosted MCP server gives Claude, Cursor and other MCP clients the same search and fetch tools with no code. Point the client at https://mcp.usestring.ai/v2/mcp (MCP server).

What it costs

One question answered this way is one search plus three fetches. On Growth that is $0.0016 a question: $1.00 per 1,000 searches and $0.20 per 1,000 standard fetches. On Starter it is $0.0024 (pricing).

10,000 questions a month costs $16 in usage on Growth, plus the $100 plan. A refresh of 2,000 index pages once a day is about 60,000 fetches a month: $12 on Growth, $18 on Starter at the standard rate. Pages that need premium proxies or a browser cost more: $2.00 and $1.00 per 1,000 on Growth.

The cheapest call is the one you skip. Answer from the index when it has a fresh page, search only when it does not, and re-fetch each source only as often as it changes. For more on cost per page, see how to reduce web scraping costs.

Start with fresh_context.py, a free key and one of your own questions. The first run prints how old each source is.

FAQ

How do I keep an LLM's knowledge current with live web data?

Retrieve at question time: search, fetch the top pages, read each page's date, and give the model today's date with the dated text. Keep a RAG index of your main sources fresh by re-fetching them on a schedule. On October 23, 2026, 30% of top Google results were dated after June 2026, the newest published model cutoff.

Which LLM is the most up to date?

By the vendors' own docs on October 23, 2026: Anthropic lists a June 2026 training cutoff for Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5. OpenAI lists April 30, 2026 for GPT-6 Astra. Google lists March 2026 for Gemini 3.8 Flash. A cutoff is an upper bound, and every model needs retrieval for anything newer.

Is there an AI with real-time data?

No model has real-time knowledge in its weights. Assistants with web search, and apps built on a search API, get current data by retrieving pages when you ask. The answer is only as current as the pages retrieved, so check the dates they carry.

Can an LLM be trained on live data to give real-time answers?

Not in a practical way. Training and fine-tuning take days to weeks and cost far more than retrieval, and the facts change again. Use retrieval for current facts, and fine-tuning for style, format or a fixed domain.

Will fine-tuning on the latest news add current knowledge to a model?

It adds some, slowly, and the model can still mix old and new facts with no way to say which is which. Retrieval at question time gives the model the page and its date, which it can cite. That is easier to check and to update.

How do you handle stale data in a RAG setup without reindexing everything?

Re-fetch each source on its own schedule, hash its visible text, and re-embed only the pages whose hash changed. Delete a changed page's old chunks before inserting the new ones, and store the page date with each chunk so retrieval can prefer newer ones.

How can my app pull fresh info from the web without hallucinating?

Give the model the sources, not just the question. Put today's date first, include each source's URL and page date, and tell the model to answer only from the sources, to cite them and to say when a source has no date. 49% of the top results we fetched had no date in their metadata, so that last instruction matters.

Can a local LLM access the internet?

Not by itself. A local model reaches the web through a tool your app calls: a search API and a fetch API, or an MCP server the model's client supports. The code on this page works the same with a local model; only the chat call changes.

Sources

• Freshness test, October 23, 2026: 78 questions (58 questions about web data tools, 20 time-sensitive questions) through String /v1/search (Google, US, English), the top 3 organic results fetched with /v1/fetch (format: json), one call per page; a second pass of the 20 time-sensitive questions with dateRange: "month"; a hand check of 28 pages.
• Anthropic models overview, read October 23, 2026
• OpenAI GPT-6 Astra model page, read October 23, 2026
• Gemini 3.8 Flash model card, Google DeepMind, read October 23, 2026
• Web Data Frontier Benchmark, September 16, 2026 run: 16 APIs, 100 sites, five attempts per site. Harness.
• Hacker News: npn, July 21, 2026
• Hacker News: nikcub, March 25, 2025
• Hacker News: deaux, May 20, 2026
• Hugging Face Forums: How can an LLM be trained on live data to generate real-time answers?
• schema.org dateModified, Open Graph article properties, HTML time element
• String search docs, fetch response formats, billing, MCP server, LangChain guide, AI SDK guide
• String pricing, read October 23, 2026