The EDPB’s new Guidelines 03/2026 address web scraping by private entities for generative AI training. They apply whenever personal data is involved—including “mixed” datasets—and cover the full life cycle from collection through model deployment.
Consent is effectively off the table: The guidelines confirm that consent under Art. 6(1)(a) GDPR is generally not a viable legal basis for web scraping at scale. Legitimate interest (Art. 6(1)(f)) is the primary avenue, but requires a rigorous three-part test.
Transparency obligations remain, even at scale: While individual notice may be excused under Art. 14(5)(b) where disproportionate, controllers must still publish detailed public privacy notices and facilitate data subject rights—including pre-collection opt-out mechanisms.
Robots.txt and technical signals now matter for GDPR: The EDPB treats robots.txt, ai.txt, CAPTCHAs, and login walls as relevant indicators to protect data subjects’ reasonable expectations—directly affecting the legitimate interest balancing test.
Special category data requires a life cycle approach: Incidental collection of sensitive data (Art. 9) is not automatically unlawful, but only where the controller implements concrete, demonstrable safeguards before, during, and after AI model training.
Practical safeguards tip the balance: Controllers that implement filtering, pseudonymisation, published scraping lists, opt-out tools, and anti-memorisation measures are significantly more likely to satisfy the balancing test.
Details
On 7 July 2026, the European Data Protection Board (EDPB) adopted Guidelines 03/2026 on web scraping in the context of generative AI (Version 1.0, open for public consultation). The guidelines provide the first comprehensive GDPR framework specifically addressing the large-scale extraction of publicly available data from the internet for the purpose of training generative AI models.
Web scraping—the automated extraction and storage of information from publicly accessible websites—has become a primary data sourcing technique for AI developers. The EDPB acknowledges that this processing typically occurs without data subjects’ knowledge or consent, creating significant risks to fundamental rights. The guidelines are limited to scraping by private entities and do not address data brokers who merely hold datasets without training AI themselves.
Scope and applicability
The GDPR applies whenever web scraping involves personal data—which, given the nature of internet content, is virtually always the case. The guidelines emphasise that “mixed” datasets containing both personal and non-personal data remain subject to GDPR with respect to the personal data portion.
The EDPB distinguishes between targeted scraping (restrictive criteria, e.g., specific domains or topics) and untargeted scraping or web crawling (unrestricted exploration following hyperlinks). Untargeted scraping carries higher compliance risk because the controller may have limited knowledge of what personal data is actually collected.
Controller and processor roles
Roles are determined case-by-case. The entity performing the scraping is not automatically the controller. Where an AI developer contracts a scraper to build a dataset per documented instructions, the scraper is likely a processor. Where an AI developer reuses an independently scraped dataset, each party is generally responsible only for its own processing—unless they jointly determine purposes and means, triggering joint controller status.
Legal basis: The legitimate interest test
The guidelines confirm that legitimate interest (Art. 6(1)(f) GDPR) is the most commonly relied-upon basis and requires a three-part cumulative assessment:
First, the controller must articulate a lawful, real, and precisely defined interest. The EDPB has accepted interests such as developing a conversational agent, fraud/threat detection systems, or generally developing an AI model—even before a precise end use is decided.
Second, necessity: the controller must demonstrate that scraping personal data is actually needed and that no less intrusive, equally effective alternative exists (e.g., narrower collection criteria, synthetic or pseudonymised data).
Third, the balancing test weighs the controller’s interest against data subjects’ rights and reasonable expectations. Key factors include: the “chilling effect” of indiscriminate collection on freedom of expression; the practical difficulty of exercising rights once data is scraped; the near-impossibility of removing personal data from a trained model; and data subjects’ reasonable expectations based on the source website’s nature and technical restrictions.
Reasonable expectations and technical signals
The EDPB gives significant weight to whether a source website deploys technical restrictions against scraping. Where a platform uses robots.txt, ai.txt, CAPTCHAs, or login walls—and expressly states that user data may not be used for AI training—data subjects have no reasonable expectation that their data will be scraped. Conversely, where a platform is freely accessible and explicitly informs users that content may be scraped for AI, a reasonable expectation of scraping may exist.
Practical takeaway: Ignoring robots.txt or circumventing access controls now carries direct GDPR consequences for the balancing test—not merely copyright or terms-of-service risks.
Data minimisation
The Guidelines do not prohibit training AI with large data volumes, but personal data must be adequate, relevant, and limited to what is necessary. The EDPB prescribes concrete measures at each stage: before collection (consider synthetic data, define precise criteria, exclude structurally sensitive website categories, and respect robots.txt/ai.txt); during and after collection (apply syntax-based filters for identifiers such as social security numbers, replace real data with synthetic data where feasible, and anonymise or pseudonymise).
Special category data
Processing of special categories (Art. 9 GDPR—e.g., health data, political opinions, racial or ethnic origin) is prohibited absent a specific derogation. Drawing on CJEU case law (GC & Others, C-136/17), the EDPB considers that incidental collection of special category data during AI-training scraping is not automatically unlawful—but only where the collection is purely incidental, genuinely difficult to anticipate in advance, and the controller implements lifecycle-spanning safeguards: filtering before collection, prompt deletion after collection, extraction-resistance testing during model development, and output monitoring after deployment.
Transparency at scale
Where individual notice under Art. 14 GDPR is impossible or requires disproportionate effort, the Art. 14(5)(b) exception may apply. However, this exception must not be routinely invoked. Even where individual notice is excused, the controller must publish comprehensive public information—including categories of data, purposes, legal basis, and as complete a list of scraped sources as possible (ideally, searchable domain names with collection dates). Good practice includes providing opt-out mechanisms operable even before collection begins.
Recommended next steps
Important: The Guidelines are for consultation until the end of October 2026. The final version is not expected before the end of 2026. Organizations should monitor or participate in the public consultation outcome and prepare to adjust practices when the final version is adopted. Steps to be implemented could be:
- Conduct a gap analysis of current scraping practices against the EDPB’s three-part legitimate interest test, with particular attention to the balancing factors and reasonable expectations framework.
- Review and document compliance with technical signals (robots.txt, ai.txt, CAPTCHAs) across all scraped sources, and establish processes to respect opt-out mechanisms. Implement or strengthen data minimisation measures—including pre-collection filters, syntax-based PII detection, and pseudonymisation—and document these measures for accountability purposes.
- Publish a detailed public privacy notice covering scraping activities, including source lists where feasible, and establish accessible channels for data subject rights requests. Assess exposure to special category data and implement the EDPB’s life cycle safeguards (pre-collection filtering, post-collection deletion, model-level extraction resistance, and output monitoring).