RAG on Your Own Documents and the GDPR: What the German DPAs Require

If you want to run RAG on your own documents in a GDPR-compliant way, there is one component to examine above all others: the vector index. In October 2025 the Datenschutzkonferenz (DSK, the standing conference of Germany's federal and state data protection authorities) published an 18-page guidance note on RAG systems that starts exactly there - with data minimisation before indexing, with the permission and role model inside the vector database, with the ability to delete individual entries, and with data subject rights. This article translates those requirements into technical implementation steps and a pre-launch checklist.

Why RAG on your own documents is a different GDPR question from a chatbot

As the DSK defines it, a RAG system consists of four core components: a retriever, an embedding model, a vector database and a generative language model. Before the system goes live, the reference documents are split into text segments (chunks), converted into vectors by the embedding model and stored in the vector database together with the chunks. On every query the retriever finds the semantically closest chunks and appends them to the prompt before it reaches the language model. Ideally the model contributes only linguistic ability; the factual knowledge comes from the documents.

With plain model access, the prompt, the response and the model itself are what gets examined. RAG adds a second body of data that you operate yourself - and that is the decisive difference. The DSK notes that, unlike the knowledge baked into the model during training, information in the vector database can be "updated, deleted and disclosed easily". That is both the relief and the obligation: the authorities expect you to run this data set like any other database holding personal data - with a legal basis, a permission model, retention periods and evidence.

For context: since May 2024 the DSK has published three papers on AI - the guidance note "Künstliche Intelligenz und Datenschutz" (Artificial Intelligence and Data Protection, 6 May 2024), the guidance on technical and organisational measures for developing and operating AI systems (June 2025) and, on 17 October 2025, the guidance on generative AI systems using the RAG method. The RAG guidance explicitly assumes a scenario in which the entire system runs on premise at the controller's site. As soon as parts are outsourced to processors, the DSK says, "further aspects" arise - more on that below.

What a RAG system can do in technical documentation, and where the method reaches its practical limits, is covered in AI for technical documentation. This article deals exclusively with the data protection side of Retrieval Augmented Generation in a company.

What personal data actually sits inside technical documents

The sentence "we only have technical documents, there is no personal data in there" is almost never true in practice. Technical documents are written, checked, approved and annotated by people - and those traces stay in the file. The DSK recommends cleaning headers, footers and page numbers before chunking. Headers and footers are also exactly where the inspector's name, the approval stamp and the signature live. A realistic inventory looks like this:

Two hiding places are missed again and again. First, file metadata: PDF properties carry the author, DOCX files carry a revision history with names, and both end up in the extracted text unless they are cleaned out. Second, free text: a fault entry such as "Müller retightened the terminal" is employee data, even inside a machine document. The rule of thumb for the necessity test under Art. 5(1)(c) GDPR is this: the assistant should answer what happened and how it was fixed - not who did it.

Data minimisation before indexing: what that means technically

The DSK states the requirement unambiguously in its section on data preparation: "Where personal data are not necessary, personal data are to be removed or anonymised." Where that sentence sits matters - it appears under preparation, that is, before chunking and embedding. Art. 5(1)(c) GDPR requires data to be "limited to what is necessary in relation to the purposes". Technically that means the ingest pipeline needs a redaction step, and that step runs before the embedding model ever sees the text.

1. 01

   Inventory the document classes and define an exclusion list

   Every source (DMS folder, file server share, CMMS export) is assigned to a document class. Classes with no RAG purpose - personnel files, job applications, occupational health reports, private notes - go on an exclusion list that the crawler enforces technically, not merely on paper.

2. 02

   Justify necessity per class

   For every approved class, record in writing which personal data fields the purpose genuinely requires. In technical documentation the answer is almost always: none. That justification is later your evidence under Art. 5(2) GDPR.

3. 03

   Clean headers, footers, metadata and signature images

   The extraction step strips headers, footers, page numbers, document properties and embedded signature images before the text is turned into running prose. This matches the DSK's own quality recommendation and improves retrieval quality as a side effect.

4. 04

   Detect personal data and replace it

   A combination of pattern matching (staff numbers, email addresses, phone numbers) and named entity recognition for names replaces personal fields with role placeholders such as [INSPECTOR] or [TECHNICIAN]. The technical content survives; the link to an individual disappears from the chunk.

5. 05

   Attach permission and provenance metadata

   Every chunk gets a document ID, a version, a source path, a document class and the permission groups from the source system. Without those fields, permission filters, the deletion path and access requests cannot be implemented later.

6. 06

   Only now chunk and embed - with a sample check

   Chunking and embedding run on the cleaned text. A sample per document class checks whether personal data slipped through. The result of that check is logged - as a counter per category, not as a list of the names that were removed.

The fact that a chunk is stored as a vector does not make it harmless. The DSK itself notes that it is impossible to trace why a chunk receives a particular embedding or what the vector means - but opacity is not protection. Under LLM08:2025 ("Vector and Embedding Weaknesses") OWASP lists embedding inversion attacks, which recover source information from vectors. Purpose limitation adds to this: the DSK warns that personal data from the vector database can be combined with data held inside the language model without that being visible in the output. What is not in the index cannot be combined.

Permissions in the vector index - the most frequently missed point

A vector index without a permission filter makes every indexed document findable by anyone who asks - semantic search knows nothing about folder permissions. For the RAG subsystem the DSK requires a permission and role model that covers "access restrictions to areas of the vector database and to the reference documents", plus tenant separation or functional separation of the data. It points out that this cannot be controlled inside the language model itself - but it can be controlled in the index. OWASP describes the same problem under LLM08:2025 as unauthorised access to embeddings and leakage between user groups, and recommends "permission-aware vector and embedding stores". There are three ways to implement permissions in a vector index:

Permissions must not be reinvented inside the RAG system. They come from the source system - the access lists of the DMS, the file server or the maintenance system - and are written onto the chunk at ingest. If a permission changes in the source system, the index has to follow, either by event or by a delta run at a defined interval. An employee who moves to another department must lose access to the old documents in the assistant within that interval. You verify this with negative tests: an account with no rights to a known document asks pointed questions about it and must get no hit.

Erasure, rectification and access requests with embeddings

Deleting from a vector index is feasible because every entry is individually addressable - the DSK names this explicitly as an advantage over the language model: "These entries are directly addressable, which means personal data can be deleted in a targeted manner." Conventional deletion routines triggered by the end of a retention period therefore work for the vector database and the reference documents alike. The precondition is a hard reference: every chunk knows its source document, that document's version and where it is stored. Without it you do not know which vectors belong to a document, and deletion turns into guesswork.

• —Erasure means removing all chunks carrying the document ID and taking the source document out of the reference set. Both in one run, and that run is logged.
• —Rectification becomes re-indexing: correct the document, delete the old chunks, embed the new version. It is the same mechanism as for any document change - it simply has to run reliably.
• —Intermediate artefacts - extracted text, OCR output, chunk files, embedding caches - belong in the deletion concept with their own retention period and are removed together with the document.
• —Response caches and prompt logs contain copies of chunks. They are part of the deletion path, otherwise the deleted paragraph lives on in the cache.
• —Backups do not delete immediately. Document the rotation period after which a deleted entry has disappeared there as well.
• —Test before go-live: ingest a test document with an invented name, delete it, then ask for it specifically - through the assistant and directly against the index.

For access requests the DSK holds that data subjects have the rights under Art. 15 et seq. GDPR "in relation to the input prompt and the output, the reference documents and the vector database", insofar as personal data are processed there. You can answer which documents and chunks the person appears in (via full-text search over chunk text and metadata, not via vector search), for what purpose, who the recipients are - an external model operator, for instance - and for how long. What an individual vector says cannot be traced, according to the DSK; that is why the chunk text must sit next to the vector or be resolvable via the document ID.

{
  "chunk_id": "c7f3…",
  "document_id": "DOC-2024-01187",
  "document_version": 3,
  "source_uri": "dms://plant2/maintenance/service-report-2024-01187.pdf",
  "doc_class": "service_report",
  "acl_groups": ["maintenance-plant-2", "engineering-management"],
  "pii_redacted": true,
  "ingested_at": "2026-09-14T06:12:00Z",
  "retention_until": "2036-09-14"
}

Clean metadata have a useful side effect: transparency. The DSK counts documenting the sources used for an answer - "references to chunks or documents" - among the measures that create traceability. In practice that means every answer carries the document ID, the version and the passage it was built from, so the reader can open the original and check it - the same metadata the permission filter and the deletion path rely on.

Logging without creating a new data protection problem

The RAG guidance itself does not prescribe any particular logging. The DSK guidance on technical and organisational measures from June 2025 counts logging among the classic IT security measures against manipulation and, in its section on data subject rights, makes clear that log data are "to be treated like conventional data". What follows from that in practice: like any other database, logs need a purpose, a retention period and an access concept. Under LLM08:2025 OWASP recommends immutable logs of retrieval activity so that misuse can be detected. The trade-off is obvious: anyone who stores every prompt and answer indefinitely builds a second body of personal data - this time about their own staff.

Keep two logs with two retention periods. The accountability log - ingest and deletion runs, permission changes, configuration states - serves the accountability duty under Art. 5(2) GDPR and is kept for a long time; it contains no prompts. The operations log - queries, retrieval, model calls - serves security and troubleshooting and gets a short period fixed in the deletion concept. Full prompt texts are stored only for troubleshooting and only briefly; access belongs to operations, not to the business unit. As early as its 2024 guidance the DSK required transparency towards employees about whether an input history is stored - so logs that would allow performance monitoring belong on the works council's table early.

Where to run it: on premise, EU cloud or hyperscaler

Where the system runs does not change what has to be done, only who does which part and to whom data are transferred. The DSK bases its assessment on fully local operation and stresses that RAG allows a language model with less factual knowledge, so the system can, in some cases, run on premise: "This can avoid personal data being transmitted to online operators of large language models." The Mittelstand-Digital Zentrum Chemnitz (a publicly funded SME digitalisation centre) makes the same point: with smaller, locally operated models there is no "security-critical transfer of data to third parties". Running AI on your own documents without the cloud avoids transfers to third parties - but it shifts the operating load, not the responsibility. Patching, permissions, deletion and logging stay with you either way.

The EU cloud is the more sensible choice when there is no GPU operating expertise in house, when load fluctuates heavily, when several sites have to be served, or when the project is still a pilot. Three conditions then apply: a processing agreement under Art. 28 GDPR - in the DSK's 2024 guidance the external provider acts as an "extended arm on behalf of the controller" (source: Orientierungshilfe Künstliche Intelligenz und Datenschutz, 6 May 2024, section on controllership) - redaction before transfer rather than after it, and a verified commitment that the provider neither stores prompts and documents nor uses them for training. What local operation actually demands in terms of hardware and model size is covered in running an LLM on premise in industry.

Pre-launch checklist

The following 14 points condense the requirements of the DSK RAG guidance and the two earlier DSK papers into a list you can tick off. It replaces neither the review by your data protection officer nor legal advice, but it makes sure both of them find something solid to work with.

Conclusion

In data protection terms, RAG on your own documents is easier to control than a language model on its own - that is precisely what the DSK says. The reason is the index: it can be cleaned, permissioned, deleted from selectively and reported on. The same property makes it the thing that gets examined. If you move data minimisation into the ingest pipeline, inherit permissions from the source system, bind every chunk hard to its document and give logging defined retention periods, you have covered the technical side of the guidance. Assessing the individual case in legal terms remains the job of your data protection officer.

Start with the inventory in Tab. 01 and the necessity test - everything else builds on them. If you are looking for a technical sparring partner to set up the ingest pipeline, the permission model and the deletion path together with your data protection officer, you will find the service under AI solutions for technical documents.