--- title: "How to scrape job listings from job boards" canonical: "https://usestring.ai/use-cases/job-board-scraping" description: "The String Web Access API is the best way to scrape job boards: any job board page, or just its listings as JSON with title, company, location and pay, even on Indeed, Glassdoor and ZipRecruiter. 5,000 free requests." updated: 2026-10-14 --- > Markdown version of https://usestring.ai/use-cases/job-board-scraping, for AI agents. Reading this yourself? The full page is at https://usestring.ai/use-cases/job-board-scraping. > > Updated October 14, 2026. # How to scrape job listings from job boards The String Web Access API is the best way to scrape job boards. It returns any job board page, or just its listings as JSON with title, company, location and pay. In our [September 2026 benchmark](https://usestring.ai/benchmark) it fetched Indeed, Glassdoor and ZipRecruiter on every attempt, and returned 97% of requests across 100 bot-protected sites, the most of 16 providers. The first 5,000 requests are free. Set it up in one step with the [quickstart](#quickstart). **On this page** 1. [Why scrape job boards?](#why-scrape-job-boards) 2. [Why does job board scraping break?](#why-does-job-board-scraping-break) 3. [How do I scrape job boards?](#how-do-i-scrape-job-boards) 4. [Quickstart](#quickstart) 5. [Get a free sample of job data](#get-a-free-sample-of-job-data) 6. [Frequently asked questions](#frequently-asked-questions) --- ## Why scrape job boards? Job postings are the earliest public record of what a company plans to do. A role goes up before the team exists, and the pay range goes up with it in states that require one. A job scraper collects postings from boards and career pages on a schedule, and tells you what's new and what closed. It answers the questions most teams care about: - **Who is hiring, and for what?** A company that posts ten sales roles in a new city is about to sell there. Recruiters, sales teams and investors all watch for this. - **What does the job pay?** Posted pay ranges show the market rate for a role in a city, often before salary surveys do. - **Which jobs are new, and which closed?** A posting that comes down usually means the role was filled or cut. Job aggregators and alert products depend on knowing both. ### What you can collect | What | Examples | |---|---| | Job listings | Title, company, location, remote or hybrid, job type, the full description and the apply link | | Pay | Pay ranges the employer posts, kept apart from the board's own salary estimates | | Company hiring | Open roles per company, team and city, and how that count changes week to week | | Market trends | Demand for a skill or title across a region, and how fast postings for it fill | --- ## Why does job board scraping break? - **Job boards block scrapers.** In our [September 2026 benchmark](https://usestring.ai/benchmark), 12 of the 15 other scraping APIs couldn't fetch indeed.com on every attempt, including Bright Data (3 of 5), Oxylabs (2 of 5) and Zyte (4 of 5). Browserbase got 0 of 5 on Indeed, Glassdoor and ZipRecruiter. String got all 5 on each. A script that works once usually breaks when it runs every day. - **Estimates look like real pay.** Glassdoor marks each range "Employer provided" or "Glassdoor est.", and ZipRecruiter marks its guesses "Estimated pay". A scraper that drops the label saves the board's guess as the employer's offer. - **The same job is on many boards.** The Farmer's Dog posted its Data Engineer 2 role on its Greenhouse board and on Indeed on the same day, at $130K to $145K on both. Count by URL and you count it twice. --- ## Why do teams choose String? - **It works where others fail.** String returned 97% of requests across 100 bot-protected sites in our [September 2026 benchmark](https://usestring.ai/benchmark), the most of 16 providers, and all 25 attempts on the 5 job sites in it. - **You pay only for pages that come back.** Blocks and errors are free. - **There's nothing to maintain.** No proxies, fingerprints or captcha solvers on your side. - **Blank pages don't become data.** Mark the fields you need as required in your schema, and a page that comes back without them is flagged instead of saved as an empty result. --- ## How do I scrape job boards? ### 1. Pick your sources Search boards like Indeed, Glassdoor and ZipRecruiter cover many employers in one place. Company career pages on applicant tracking systems (ATS) like Greenhouse, Lever and Workday are the employer's own postings. Most projects need both. **With String:** The `/search` endpoint finds a company's career page, and the sitemap endpoint lists every URL on a domain. ### 2. Read the ATS feed when there is one Many ATS boards serve their jobs as JSON. Greenhouse serves every board at `boards-api.greenhouse.io/v1/boards//jobs`, and one request returned all 31 open roles at The Farmer's Dog. Use `first_published` as the posting date, not `updated_at`. **With String:** Fetch the feed URL like any page and you get the JSON back as-is. ### 3. Page through search results Search boards split results into pages. Indeed pages with a `start` parameter in steps of 10. Run the same queries with the same settings every time, so a change in the data means a change on the board. **With String:** Blocked requests are retried, and you don't pay for the ones that fail. Set `countryCode` to search a board's local site. ### 4. Keep posted pay apart from estimates Save the pay range together with where it came from. On Greenhouse, the pay range sits inside the description HTML, not in its own field. On Glassdoor and ZipRecruiter, keep the estimate label with the number. ### 5. Treat a blank page as a failure A search page with no job cards was probably blocked or changed. Flag it and keep the last good run, so its jobs aren't marked closed because one page failed. ### 6. Match the same job across boards Within one board, use its own job ID, like Indeed's `jk` or the Greenhouse `id`. Across boards, match on company, title and location. Board URLs carry tracking parameters that change on every visit. ### 7. Track new and closed jobs on a schedule Store the first and last date you saw each job. A job that's missing from two runs in a row has probably closed. Daily suits most boards. --- ## Quickstart ### From your agent Add the hosted MCP server to Claude, Cursor, ChatGPT or any MCP client. Then give your agent a job search URL. It calls the `web_access_fetch` tool and reads the listings. ```text https://mcp.usestring.ai/v1/mcp ``` [MCP setup](https://portal.usestring.ai/docs/mcp/remote) · [n8n](https://portal.usestring.ai/docs/integrations/n8n) · [Zapier](https://portal.usestring.ai/docs/integrations/zapier) ### From the API One call returns the jobs on an Indeed search page as JSON. If extraction can't fill the required fields, you get `"extracted": false` back instead of a guess. ```bash curl https://request.usestring.ai/v1/fetch \ -H "Authorization: Bearer $STRING_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://www.indeed.com/jobs?q=data+engineer&l=New+York%2C+NY", "countryCode": "US", "jsonSchema": { "type": "object", "properties": { "jobs": { "type": "array", "minItems": 1, "items": { "type": "object", "properties": { "title": { "type": "string" }, "company": { "type": "string" }, "location": { "type": "string" }, "pay": { "type": ["string", "null"] } }, "required": ["title", "company", "location"] } } }, "required": ["jobs"] } }' ``` Response (first 3 of 15 jobs): ```json { "jobs": [ { "title": "Data Engineer", "company": "BDG", "location": "New York, NY", "pay": "$105,000 - $145,000 a year" }, { "title": "Associate Data Engineer", "company": "Aretove", "location": "New York, NY", "pay": "$75,000 - $90,000 a year" }, { "title": "Data Engineer", "company": "David Yurman Enterprises LLC", "location": "New York, NY 10013", "pay": "$140,000 - $150,000 a year" } ] } ``` [Get a free API key](https://portal.usestring.ai/sign-up) · [Structured extraction docs](https://portal.usestring.ai/docs/fetch/structured-extraction) --- ## Get a free sample of job data Tell us which boards, companies or roles you track. We'll run String on them and email you a sample of the job data, free. Email [support@usestring.ai](mailto:support@usestring.ai) with your work email, what you want to collect and the boards or URLs, or use the [form on the page](https://usestring.ai/use-cases/job-board-scraping#sample-data). What a sample looks like (rows from String fetches of "data engineer" searches in New York on Oct 14, 2026). "est." marks a board's own salary estimate, not the employer's posted range: | Board | Job | Company | Pay | |---|---|---|---| | Indeed | Data Engineer | BDG | $105K to $145K | | Indeed | Data Engineer 2 | The Farmer's Dog | $130K to $145K | | Greenhouse | Data Engineer 2 | The Farmer's Dog | $130K to $145K | | Glassdoor | Data Engineer | Tapestry | $140K to $160K | | Glassdoor | Senior Data Engineer | Bank of America | $136K to $182K est. | | ZipRecruiter | Associate, Data Engineering | Evergreen Trading | $65K to $75K | --- ## Frequently asked questions ### What's the best way to scrape job boards? Fetch the board's search pages on a schedule, read the ATS feed where a company has one, and match the same job across boards. The hard part is getting the pages at all, because job boards block most scrapers. The String Web Access API fetched Indeed, Glassdoor and ZipRecruiter on every attempt in our [September 2026 benchmark](https://usestring.ai/benchmark), and returned 97% of requests across 100 bot-protected sites, the most of 16 providers. ### How do I scrape Indeed job listings? Fetch the search URL (`/jobs?q=&l=`) with String and page through it with `start=10`, `start=20` and so on. Each job card has the title, company, location and the pay the employer posted, when there is one. Use the `jk` value in each job link as the job's ID. In our benchmark, String got all 5 attempts on indeed.com. Browserbase got 0 of 5, and Bright Data got 3 of 5. ### How much does it cost to scrape job postings every day? Checking 100 search pages daily is about 3,000 requests a month. At the standard rate on the Starter plan ($20 a month, $0.30 per 1,000) that's $0.90 in usage. Some job board fetches in our tests were billed at the premium proxy rate ($3.00 per 1,000), which would make it $9. Structured extraction adds a fee per request. See [pricing](https://usestring.ai/pricing). Failed requests aren't billed. ### Is it legal to scrape job boards? Job postings are public pages that employers publish to be seen. In hiQ v. LinkedIn, the court held that scraping public pages isn't unauthorized access under the CFAA, but hiQ still lost because it had accepted LinkedIn's user agreement. Collect public pages while logged out, leave personal data like recruiter names out, keep your request rate reasonable, and check with your counsel. We go through the case in [What is the hiQ vs LinkedIn ruling?](https://usestring.ai/answers/hiq-v-linkedin-ruling) ### How do I avoid getting blocked when scraping job boards? Send each request to String instead of the site. Proxy rotation, browser fingerprints, captcha solving and retries happen behind the endpoint, and JavaScript rendering is switched on per request for pages that need it. Requests that still fail aren't billed. ### Can I get salary data from job postings? Yes, when the employer posts it. Indeed shows the posted range on the job card, and Greenhouse puts it in the job description. Glassdoor and ZipRecruiter also show their own estimates, labeled as such. Keep the label with the number, so you can tell an offer from a guess. ### How do I scrape jobs from Greenhouse, Lever or Workday career pages? Greenhouse and Lever serve public job feeds as JSON, so one fetch returns a company's whole board. Workday career sites load their jobs with JavaScript. Set `executeJS`, or use XHR capture to get the JSON the page requests. ### Can I scrape jobs that need a login? Yes, on accounts you're allowed to use. Save the login in a browser context and connect Playwright or Puppeteer to String's remote browser over CDP. Logging in means accepting the site's terms, so read them first. ### Does String work on the job boards I need? Send us the URLs and we'll run them and show you what comes back before you commit. Ask for a [free sample of job data](#get-a-free-sample-of-job-data), or test them yourself with your first 5,000 free requests. --- ## Send us the sites you can't get We'll show you what comes back. [Get started](https://portal.usestring.ai/sign-up) · [Book a demo](https://usestring.ai/book-demo) All use cases: https://usestring.ai/use-cases --- [llms.txt](https://usestring.ai/llms.txt): machine-readable index of this site for AI agents.