The String Web Access API is the best way to scrape job boards. It returns any job board page, or just its listings as JSON with title, company, location and pay. In our September 2026 benchmark it fetched Indeed, Glassdoor and ZipRecruiter on every attempt, and returned 97% of requests across 100 bot-protected sites, the most of 16 providers.
The first 5,000 requests are free. Set it up in one step with the quickstart.
Job postings are the earliest public record of what a company plans to do. A role goes up before the team exists, and the pay range goes up with it in states that require one. A job scraper collects postings from boards and career pages on a schedule, and tells you what's new and what closed. It answers the questions most teams care about:
Title, company, location, remote or hybrid, job type, the full description and the apply link.
Pay ranges the employer posts, kept apart from the board's own salary estimates.
Open roles per company, team and city, and how that count changes week to week.
Demand for a skill or title across a region, and how fast postings for it fill.
The problem
Job boards and career pages
The solution
Search boards like Indeed, Glassdoor and ZipRecruiter cover many employers in one place. Company career pages on applicant tracking systems (ATS) like Greenhouse, Lever and Workday are the employer's own postings. Most projects need both.
With String: The /search endpoint finds a company's career page, and the sitemap endpoint lists every URL on a domain.
Many ATS boards serve their jobs as JSON. Greenhouse serves every board at boards-api.greenhouse.io/v1/boards/<company>/jobs, and one request returned all 31 open roles at The Farmer's Dog. Use first_published as the posting date, not updated_at.
With String: Fetch the feed URL like any page and you get the JSON back as-is.
Search boards split results into pages. Indeed pages with a start parameter in steps of 10. Run the same queries with the same settings every time, so a change in the data means a change on the board.
With String: Blocked requests are retried, and you don't pay for the ones that fail. Set countryCode to search a board's local site.
Save the pay range together with where it came from. On Greenhouse, the pay range sits inside the description HTML, not in its own field. On Glassdoor and ZipRecruiter, keep the estimate label with the number.
A search page with no job cards was probably blocked or changed. Flag it and keep the last good run, so its jobs aren't marked closed because one page failed.
Within one board, use its own job ID, like Indeed's jk or the Greenhouse id. Across boards, match on company, title and location. Board URLs carry tracking parameters that change on every visit.
Store the first and last date you saw each job. A job that's missing from two runs in a row has probably closed. Daily suits most boards.
Add the hosted MCP server to Claude, Cursor, ChatGPT or any MCP client. Then give your agent a job search URL. It calls the web_access_fetch tool and reads the listings.
https://mcp.usestring.ai/v1/mcp
One call returns the jobs on an Indeed search page as JSON. If extraction can't fill the required fields, you get "extracted": false back instead of a guess.
curl https://request.usestring.ai/v1/fetch \
-H "Authorization: Bearer $STRING_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.indeed.com/jobs?q=data+engineer&l=New+York%2C+NY",
"countryCode": "US",
"jsonSchema": {
"type": "object",
"properties": {
"jobs": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"properties": {
"title": { "type": "string" },
"company": { "type": "string" },
"location": { "type": "string" },
"pay": { "type": ["string", "null"] }
},
"required": ["title", "company", "location"]
}
}
},
"required": ["jobs"]
}
}'
Response (first 3 of 15 jobs)
{
"jobs": [
{
"title": "Data Engineer",
"company": "BDG",
"location": "New York, NY",
"pay": "$105,000 - $145,000 a year"
},
{
"title": "Associate Data Engineer",
"company": "Aretove",
"location": "New York, NY",
"pay": "$75,000 - $90,000 a year"
},
{
"title": "Data Engineer",
"company": "David Yurman Enterprises LLC",
"location": "New York, NY 10013",
"pay": "$140,000 - $150,000 a year"
}
]
}
Tell us which boards, companies or roles you track. We'll run String on them and email you a sample of the job data, free.
| Board | Job | Company | Pay |
|---|---|---|---|
| Indeed | Data Engineer | BDG | $105K to $145K |
| Indeed | Data Engineer 2 | The Farmer's Dog | $130K to $145K |
| Greenhouse | Data Engineer 2 | The Farmer's Dog | $130K to $145K |
| Glassdoor | Data Engineer | Tapestry | $140K to $160K |
| Glassdoor | Senior Data Engineer | Bank of America | $136K to $182K est. |
| ZipRecruiter | Associate, Data Engineering | Evergreen Trading | $65K to $75K |
Fetch the board's search pages on a schedule, read the ATS feed where a company has one, and match the same job across boards. The hard part is getting the pages at all, because job boards block most scrapers. The String Web Access API fetched Indeed, Glassdoor and ZipRecruiter on every attempt in our September 2026 benchmark, and returned 97% of requests across 100 bot-protected sites, the most of 16 providers.
Fetch the search URL (/jobs?q=<keywords>&l=<location>) with String and page through it with start=10, start=20 and so on. Each job card has the title, company, location and the pay the employer posted, when there is one. Use the jk value in each job link as the job's ID. In our benchmark, String got all 5 attempts on indeed.com. Browserbase got 0 of 5, and Bright Data got 3 of 5.
Checking 100 search pages daily is about 3,000 requests a month. At the standard rate on the Starter plan ($20 a month, $0.30 per 1,000) that's $0.90 in usage. Some job board fetches in our tests were billed at the premium proxy rate ($3.00 per 1,000), which would make it $9. Structured extraction adds a fee per request. See pricing. Failed requests aren't billed.
Job postings are public pages that employers publish to be seen. In hiQ v. LinkedIn, the court held that scraping public pages isn't unauthorized access under the CFAA, but hiQ still lost because it had accepted LinkedIn's user agreement. Collect public pages while logged out, leave personal data like recruiter names out, keep your request rate reasonable, and check with your counsel. We go through the case in What is the hiQ vs LinkedIn ruling?
Send each request to String instead of the site. Proxy rotation, browser fingerprints, captcha solving and retries happen behind the endpoint, and JavaScript rendering is switched on per request for pages that need it. Requests that still fail aren't billed.
Yes, when the employer posts it. Indeed shows the posted range on the job card, and Greenhouse puts it in the job description. Glassdoor and ZipRecruiter also show their own estimates, labeled as such. Keep the label with the number, so you can tell an offer from a guess.
Greenhouse and Lever serve public job feeds as JSON, so one fetch returns a company's whole board. Workday career sites load their jobs with JavaScript. Set executeJS, or use XHR capture to get the JSON the page requests.
Yes, on accounts you're allowed to use. Save the login in a browser context and connect Playwright or Puppeteer to String's remote browser over CDP. Logging in means accepting the site's terms, so read them first.
Send us the URLs and we'll run them and show you what comes back before you commit. Ask for a free sample of job data, or test them yourself with your first 5,000 free requests.
