Scraping publicly available data is generally lawful in the United States, and the courts that matter have said so repeatedly. What creates real legal exposure is rarely the act of fetching a page. It is the four things around it: the contract you agreed to, whether the data is personal, whether the content is copyrightable, and whether you had to log in or defeat an access control to get it. A public product listing read by an anonymous request sits at one end of that range. A logged-in profile pulled behind an account you agreed to terms on sits at the other.
This page describes how courts have ruled. It is not legal advice, and scraping law differs by jurisdiction and moves quickly. Talk to counsel about your specific target list.
The federal hacking statute does not cover public pages. In hiQ Labs v. LinkedIn, the Ninth Circuit twice held that scraping data available to the general public does not amount to access "without authorization" under the Computer Fraud and Abuse Act, because public data has no gate to get past. The Supreme Court's decision in Van Buren v. United States (2021) pointed the same way, reading the CFAA through a "gates-up-or-down" test rather than a rules-violation test.
Contract law is where scrapers actually lose. The same hiQ case makes the point better than any commentary. hiQ won on the CFAA and then lost on breach of contract: in November 2022 the district court granted LinkedIn summary judgment on its user-agreement claim. The case settled in December 2022 with hiQ accepting a permanent injunction, $500,000 in damages, and an order to delete the data and the code that collected it. The company that established the leading pro-scraping precedent was put out of the scraping business by a contract claim.
But terms only bind you if you agreed to them. In Meta Platforms v. Bright Data (January 2024), Judge Edward Chen granted Bright Data summary judgment. The reasoning is the practical rule most teams need: when Bright Data scraped while logged out, it was not a "user" of Facebook or Instagram and so was not bound by the terms of service. The court also held that Meta's survival clause, which purported to bar scraping of public data forever after account termination, was unenforceable.
Read together, those three cases give a usable line. Logged-out collection of public pages is defensible. Collection that depends on an account, or that runs against terms you actually accepted, is where liability lives.
Personal data. GDPR applies to personal data about people in the EU no matter where you found it, and "it was public" is not an exemption. That is the theory behind the Clearview AI enforcement actions across several European regulators. US law is more permissive: the CCPA, as amended by the CPRA, excludes information lawfully made available to the general public from its definition of personal information, and it is an opt-out regime rather than a consent regime.
Copyright. Facts are not copyrightable (Feist v. Rural Telephone), which is why prices, specifications, and other raw data points are usually safe to extract. The expression around them can be protected, and reproducing full articles is a different act from extracting a number. Whether training a model on copyrighted text is fair use is being litigated now and should not be treated as settled.
robots.txt. It is a voluntary protocol, not a statute. Ignoring it creates no direct liability on its own, but it can serve as evidence of notice in a contract or trespass claim.
String's services collect publicly available web data and turn it into datasets, analytics, and monitoring, and customer-directed requests stay subject to our acceptable-use requirements. Our privacy policy states that we do not sell personal information, do not share request data between customers, and do not train models on customer request content. Our crawler policy is published in our robots.txt, and our current compliance status, including where a SOC 2 report does not yet exist, is on the trust center.
None of that is a statement about your legal position. What you may collect depends on your target list, your jurisdiction, and your own counsel's read. Using a vendor does not transfer your legal risk, and we will not tell you otherwise.
Generally yes. The Ninth Circuit held in hiQ Labs v. LinkedIn that scraping public data does not violate the Computer Fraud and Abuse Act, and the Supreme Court's Van Buren decision supports that reading. The exposure comes from contract terms you accepted, from privacy law when the data is personal, and from copyright when you copy protected expression rather than facts.
Yes, and the claim is usually breach of contract rather than hacking. hiQ Labs won its CFAA fight with LinkedIn and still ended up under a permanent injunction, paying $500,000 and deleting its data, because it lost the contract claim. Trespass to chattels, copyright, and privacy claims also appear. Being right on the CFAA is not the same as being safe.
Only against someone bound by those terms. In Meta v. Bright Data the court found Bright Data was not bound while scraping logged out, because it was not then a "user" of the service, and it struck down a survival clause that tried to extend the ban past account termination. Terms accepted through an account you actually use are a much stronger claim for the site.
No. robots.txt is a voluntary convention standardised as RFC 9309, not law, and no US court has held that ignoring it is by itself unlawful. It still matters: a site can point to it as evidence you were on notice, which can strengthen a contract or trespass claim against you.
Unsettled, and moving. Collection and use are separate questions, and the fair-use analysis for training on copyrighted expression is being actively litigated in several US cases. Treat any confident answer in either direction with suspicion, and get advice specific to your corpus.
No. A vendor can shape how collection happens and honour published crawler policies. The decision about which pages to collect and what to do with the result stays with you, and so does the liability. Our terms of service set out what we require of customers; they do not substitute for your own legal advice.
