Generative AI and the GDPR

Most generative AI deployments inside an organisation are not new processing operations. They are an additional, much wider route of access to processing operations that already exist. That is why the Dutch Data Protection Authority’s practical tool for deploying generative AI starts not with the model but with the record of processing activities and the authorisation policy: if you cannot say which personal data you hold and who may reach it, you cannot say what a system granted the user’s own file access is doing with it.

This page is guidance, not legal advice. Where it states an obligation it names the article, and where the regulator’s tool declines to reach a conclusion (it deliberately does not address lawful basis), so does this page.

Where the personal data actually is

Teams tend to assess “does the model see personal data?” and stop at the prompt. The processing surface is wider, and each part of it has its own storage location and retention period.

The AP’s tool is explicit that the categories whose storage locations and retention you must control include, in addition to input and output, metadata, diagnostic data and telemetry data. It is equally explicit about the retrieval surface: some integrated systems have access to all files the user can access, and if the system reaches all data in the organisation, complying with data minimisation is difficult, because it is unlikely all of that data is necessary for the specific purpose.

That is the central practical finding. The privacy problem with an assistant bolted onto an existing collaboration suite is rarely the model. It is that years of over-broad file permissions, which were survivable when access required a human to go looking, become instantly exploitable by a system that searches everything the user may see in one shot.

Lay the foundation first

The tool’s first chapter is about what must be true before the choice of system is discussed.

Record of processing activities (Article 30). Check it, update it, and use it to work out which existing processing operations the deployment touches. The new processing must also be entered in it.

An inventory of personal data. Which categories you process and in which applications. You must be able to state that personal data is only processed in the places mapped, including the awkward cases, such as email.

Authorisation management. Not every employee should reach all personal data, including through the AI system. Authorisations must match roles, and the policy must be implemented in practice rather than documented. Pay attention to dynamic authorisations in the underlying applications.

Cloud context. Where the system is part of an existing cloud estate, there must be a data processing agreement with that provider, and you must have investigated whether the standard agreement needs amending for generative AI specifically. Deploying the system can raise the risk of existing processing operations, which is a different question from the risk of the new one.

Two framing points from the tool are worth keeping: the AP recommends organisations first explore whether generative AI can be deployed without personal data at all; and the use of other services from a supplier is not sufficient reason to also choose that supplier’s AI system.

Purpose limitation, before anything else

Under Article 5(1)(b) personal data may be collected only for specified, explicit and legitimate purposes. The AP’s application of this to generative AI is unusually direct: deploying a generative AI system must not be a purpose in itself; the system should help solve a concrete problem. Purposes that are unclear, too broad, or expressed in technical terms lead to unlawful processing, and the tool names “supporting staff” and “making a process more efficient” as descriptions that fail the test.

The second half of the obligation is the one most deployments skip: you must prevent the system being used for other purposes. In practice that means internal policy on what the tool may be used for, enforced by configuration where possible, not an acceptable-use paragraph nobody reads.

Lawful basis, and why legitimate interest is harder than assumed

Article 6(1) lists six bases, and processing is lawful only to the extent at least one applies. The AP’s practical tool does not address lawful basis at all, because it derives from a prior consultation under Article 36 that did not cover the question. Nothing below is therefore the regulator’s conclusion on this point. It is what Article 6 says.

Article 6(1)(f) permits processing necessary for the legitimate interests of the controller or a third party, “except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject”. Three parts of that wording do the work.

Necessary is a real constraint. If the purpose can be achieved with less personal data, or none, the processing is not necessary for it, and the AP’s recommendation to first explore deployment without personal data bites here.

The balancing test is against data subjects who, in a typical internal deployment, are customers, patients, applicants or citizens whose files the assistant can read, not the employees using it. They have no relationship with the AI supplier and no expectation that their file will be summarised by a third-party model. The AP notes that a system which shares personal data with the supplier and uses it for further model training has a greater impact on data subjects than a self-hosted one.

The public-authority carve-out. Article 6(1) states that point (f) does not apply to processing carried out by public authorities in the performance of their tasks.

Special categories of data under Article 9 need a separate condition on top of the Article 6 basis. A system with unrestricted access to an HR or clinical file share will encounter Article 9 data whether or not the deployment intended it to.

The DPIA trigger

Article 35(1) requires a DPIA before processing where a type of processing, “in particular using new technologies”, is likely to result in a high risk to rights and freedoms. Article 35(3) lists three cases where one is required in particular: systematic and extensive automated evaluation producing legal or similarly significant effects; large-scale processing of Article 9 data or Article 10 criminal-offence data; and systematic large-scale monitoring of a publicly accessible area.

The AP’s position on generative AI: a DPIA is needed when a new technology is implemented in data processing, such as generative AI, and a pilot can also process personal data and therefore also needs one. Three further constraints from the tool:

• You must carry out the DPIA yourself. Pointing at another organisation’s generic or reference DPIA is not sufficient.
• The Data Protection Officer must be asked for advice on the DPIA (Article 35(2)) and, per Article 38(1), should be involved in the whole process at an early stage.
• If high residual risks remain after the DPIA, prior consultation with the supervisory authority is mandatory under Article 36.

Completing the AP’s tool is not a DPIA and does not replace one; the tool says so itself, and suggests attaching it to the DPIA instead.

Data subject rights against a deployed model

Rights are where the architecture and the law diverge most sharply, and where dependence on the supplier is total. The AP’s framing is that as a deployer you are largely dependent on the information the supplier provides, and that if that transparency is missing, your organisation cannot comply with its obligation to facilitate data subject rights.

Article 17 gives a right to erasure without undue delay on any of six grounds, including that the data are no longer necessary for the purposes, that consent has been withdrawn with no other ground, that the data subject has objected under Article 21(1) with no overriding legitimate grounds, and that the data were unlawfully processed. Article 17(2) adds that where the controller has made the data public, it must take reasonable steps - “taking account of available technology and the cost of implementation” - to inform other controllers processing it of the erasure request.

Deleting a conversation record is straightforward. Removing the influence of personal data used to train or fine-tune a model is not: the data are not stored as records but distributed across parameters, and there is no established, verifiable deletion operation short of retraining. Article 17 contains no exemption for that difficulty. The workable conclusion is architectural: do not put personal data into training or fine-tuning corpora unless you can retrain, and contract explicitly on the point. The AP’s tool asks exactly that: whether you have made agreements with the supplier about the use of personal data for further training, and whether those parties may use your shared data to train their own model.

Article 22(1) gives a right not to be subject to a decision based solely on automated processing, including profiling, producing legal effects or similarly significant effects. The exceptions in Article 22(2) are narrow: necessity for a contract, authorisation by Union or Member State law with safeguards, or explicit consent. In the contract and consent cases Article 22(3) still requires at least the right to obtain human intervention, to express a point of view, and to contest the decision.

The AP treats this as a design problem. Check whether individual decision-making occurs in the work processes where generative AI is used; if so, ensure the human assessment remains decisive, that users know enough about the system’s functioning to disagree with it, and that they are authorised to deviate from the output. The observation that carries most weight operationally: the less time an employee has, the less meaningful the human intervention is. A rubber-stamp review under time pressure is not the safeguard Article 22(3) contemplates.

Controller, processor, and the supplier contract

The tool requires you to establish the role of every involved party before contracting: are they controller, joint controller, or processor? If a third party is a processor, a data processing agreement is required.

Article 28(3) sets what that agreement must contain. The processor must act only on documented instructions from the controller; ensure confidentiality commitments from authorised persons; take the Article 32 security measures; not engage a sub-processor without written authorisation, the initial processor remaining fully liable for its performance; assist the controller with data subject rights requests and with Articles 32 to 36; delete or return the personal data at the end of the service and delete existing copies unless law requires retention; make available all information necessary to demonstrate compliance, including submitting to audits; and inform the controller immediately if it considers an instruction to infringe the Regulation.

The friction with a third-party model API is concentrated in three of those clauses. “Only on documented instructions” is inconsistent with a supplier reusing your data for its own model improvement. That would make it a controller for that purpose, not a processor. “Deletes or returns all the personal data” is the clause that collides with training. And the sub-processor rule is non-trivial when the model provider itself runs on someone else’s infrastructure.

The AP’s tool also puts the burden squarely back on the deployer for retention: even if the supplier does not provide clarity about retention periods or does not allow them to be changed, your organisation remains obliged to comply with the GDPR, and it states plainly that this can obstruct the deployment. That is the sentence to quote when a procurement process treats an opaque retention policy as an acceptable residual risk.

The checks the regulator’s tool specifies

The tool is a set of statements, each answered agree or disagree with a written explanation, because the explanation is what discharges accountability under Article 5(2) and Article 24. It should be completed by privacy professionals, IT specialists and the business units that want the system, with the DPO involved early. It covers deployment only, and organisations developing their own models are directed to the AP’s separate guidance. It does not address the AI Act, Regulation (EU) 2024/1689.

Two of these deserve emphasis because they are technical controls, not paperwork.

Real-time input filters and output filters are named by the tool as risk-mitigating functionality to weigh when choosing a system: output filters reduce the risk of the model generating undesired answers or special category data; input filters alert users when they are entering personal data. Choosing a system partly on whether it has them is a privacy-by-design decision under Article 25(1), which the tool notes applies to the choice of the means of processing, not only to software you build. There is a related exposure: the model repeating personal or confidential data to a user who should not see it. That is covered in sensitive information disclosure.

Accuracy (Article 5(1)(d)) conflicts structurally with how these systems work. The AP’s explanation is worth stating precisely: the system does not reason with knowledge of the world but predicts which words fit best in a given context, so an incorrect answer can look logical and convincing, and errors in training data can be repeated. Incorrect personal data must be erased or rectified without delay, which is an obligation on generated output, not only on stored records.

Periodic reassessment closes the loop. Because these systems are deeply integrated and the processing around them keeps changing, without periodic review the system drifts outside the frameworks originally set for the data processed, the purposes, and the retention periods applied. The register of processing activities is the instrument for that review.

For the wider control set this fits into, see AI governance.