Digital Provenance for Training Data Used in GxP Decisions

Why the Provenance Record Became a Deliverable

A GxP decision is a decision that touches patient safety, product quality, or data integrity. Releasing a batch. Accepting or rejecting a visual inspection result. Classifying a deviation. Flagging an adverse event for expedited reporting. When a trained model participates in any of those decisions, the company has to be able to explain the decision. Explaining the decision means explaining the model. Explaining the model means explaining the data it learned from.

That last step is where most programs run out of documentation. A model registry entry will usually record the algorithm, the hyperparameters, the validation metrics, and the approval signatures. It will point at a training dataset by name. Then the trail stops. The dataset name refers to a folder on a network share, or a table in a lake, or an export a data scientist pulled eighteen months ago from a system that has since been upgraded twice. Nobody wrote down which query produced it, which rows were dropped, which columns were derived, or who decided that the twelve thousand images labeled “acceptable” were in fact acceptable.

This is not a hypothetical weakness. The broader machine learning field has measured it. A large-scale audit of more than 1,800 widely used text datasets found license omission rates above 70 percent and license error rates above 50 percent on popular dataset hosting sites, which the authors described as a crisis in misattribution and informed use of the datasets behind many recent advances.⁸ A separate longitudinal audit of 14,000 web domains found that within a single year roughly 45 percent of one major training corpus had become restricted by website terms of service, and more than 28 percent of its most actively maintained sources had been fully restricted through robots.txt.⁹ Data that was fairly used at collection time can become data you no longer have clear rights to.

Pharma and biotech teams often assume these findings apply only to companies training large public models. They apply to any team that assembled a training set from more than one source, which is nearly every team. A visual inspection model trained on images from three manufacturing sites, a deviation classifier trained on ten years of quality records including records inherited through an acquisition, a pharmacovigilance triage model trained partly on a licensed literature corpus: each of these has a rights question, a lineage question, and a version question. Answering them after the fact is expensive and often impossible.

What the Rules Actually Say Today

Before designing the record, it is worth being precise about what is legally binding right now, because a good deal of published commentary is not. Verified against primary sources as of September 2026, the position is as follows.

The EU AI Act: data governance duties for high-risk systems are not yet in force

Regulation (EU) 2026/1744, the Digital Omnibus on AI, was adopted on 8 July 2026, published in the Official Journal on 24 July 2026, and entered into force on 27 July 2026.¹ It defers the application of the AI Act’s obligations for standalone high-risk systems listed in Annex III from 2 August 2026 to 2 December 2027, and for AI embedded in products already covered by EU product safety law under Annex I to 2 August 2028.¹ The Article 10 data governance duties that sit inside that high-risk regime move with it.

The general-purpose AI provider obligations did not move. The prohibited practices regime was expanded rather than deferred: the existing prohibitions still apply from 2 February 2025, and new prohibitions added by the same regulation apply from 2 December 2026.¹ The second of those matters here for a practical reason: the European Commission published an Explanatory Notice and Template for the Public Summary of Training Content for general-purpose AI models on 24 July 2025.¹² That template is the closest thing available to a regulator’s own view of what a training data disclosure should contain. It asks for model and provider identification, an organized listing of the main data source categories used in training (public datasets, licensed datasets, crawled or scraped online content, user data, synthetic data, other sources), and a processing and governance section covering copyright measures, removal of illegal content, and data protection compliance.¹² A pharma team designing an internal provenance record can borrow that structure directly even though the obligation itself does not apply to them.

21 CFR Part 11: the record must be retrievable, controlled, and change-tracked

Part 11 does not say what a provenance record contains. It says what happens to it once it exists. Section 11.10(b) requires the ability to generate accurate and complete copies of records in both human readable and electronic form suitable for inspection, review, and copying by the agency. Section 11.10(c) requires protection of records to enable accurate and ready retrieval throughout the retention period. Section 11.10(e) requires secure, computer-generated, time-stamped audit trails that independently record operator entries and actions that create, modify, or delete electronic records, and requires that record changes not obscure previously recorded information. Section 11.10(k)(2) requires revision and change control procedures that maintain an audit trail documenting time-sequenced development and modification of systems documentation.²

Read those four together and the design implications are unusually concrete. The provenance record has to be exportable in a form a person can read and a system can process. It has to survive as long as the retention obligation on the decisions the model influenced. Changes to it have to be appended rather than overwritten. And the sequence of changes has to be reconstructable. That rules out a wiki page. It rules out a spreadsheet on a shared drive. It largely rules out anything that is not under formal document or record control.

EU GMP: Annex 11 governs, draft Annex 22 concentrates on test data

Annex 11 of the EU GMP guide covers computerised systems generally and is the operative text for GxP systems in Europe. The draft Annex 22 on artificial intelligence, released for stakeholder consultation on 7 July 2025 with the consultation period closing on 7 October 2025, is explicitly positioned as additional guidance to Annex 11 for computerised systems in which AI models are embedded.³ It remains a consultation draft. There is no final adopted text and no announced implementation date. Any internal plan that assumes an effective date is assuming something the regulators have not said.

The draft’s content is worth reading closely, because it shapes what a provenance record needs to prove. Section 5 sets out expectations for test data: it should be representative of and expand the full sample space of the intended use, be stratified, include all subgroups, and reflect the limitations, complexity, and common and rare variations of that intended use, with the criteria and rationale for selection documented. Test datasets and subgroups should be sufficient in size to calculate test metrics with adequate statistical confidence. Labeling of test data should be verified through a process that ensures a very high degree of correctness, which may include independent verification by multiple experts, validated equipment, or laboratory tests. Any pre-processing should be pre-specified with a rationale. Any cleaning or exclusion should be documented and fully justified. Generation of test data or labels by means of generative AI is not recommended and any use should be fully justified.³

Section 6 goes further on independence: technical or procedural controls should ensure data used to test a model was not used during development, training, or validation; test data should be protected by access control and audit trail functionality logging accesses and changes; there should be no copies of test data outside that repository; and it should be recorded which data has been used for testing, when, and how many times.³

Industry guidance: structure without a schema

The ISPE GAMP Guide: Artificial Intelligence, announced on 29 July 2025, is the first comprehensive AI guidance aimed at the pharmaceutical industry. It positions data as the backbone of AI-enabled systems while keeping patient safety, product quality, and data integrity at the front of decision making, and builds on GAMP 5 Second Edition, the Records and Data Integrity Guide, and the Data Integrity by Design Good Practice Guide.⁴ It gives you the lifecycle framing and the vocabulary. It does not give you a field list either.

The EMA reflection paper on the use of artificial intelligence in the medicinal product lifecycle, adopted in September 2024, is the clearest European statement of principle: sources of data and any processing activity should be documented in detail to allow traceability in line with GxP requirements, and bias identification is treated as a high priority.⁵ Again, a principle rather than a schema.

One more useful signal comes from outside life sciences. The NIST Generative AI Profile, published as NIST AI 600-1, treats content provenance as one of four primary considerations alongside governance, pre-deployment testing, and incident disclosure.¹³ Worth noting that NIST’s use of the term is mostly about the provenance of AI outputs and synthetic content, which is a different problem from the provenance of training inputs. Teams that borrow the vocabulary without noticing the distinction end up building the wrong control.

The Provenance Record, Field by Field

What follows is a working schema. It is organized into five blocks that answer five different questions, and it is deliberately flat enough to be produced by a pipeline rather than written by a person. The identifiers matter: every block hangs off a dataset version identifier that is immutable once issued.

Block 1: Source

One entry per contributing source. A training set assembled from four systems has four source entries, not one. This is where most reconstructions fail, because the merged dataset has lost the boundary between its inputs.

Block 2: Rights and license

One entry per source, and it needs a status rather than a free text note. The audit findings on license omission and misattribution cited above are what happens when this block is optional.⁸

The field that catches teams out is permitted_use. A vendor agreement that permits a company to analyze its own instrument data does not automatically permit training a model on it and deploying that model at other sites. A literature license that permits reading does not automatically permit ingestion. These are different rights and they need separate confirmation.

Block 3: Transformation

Every step between the raw extract and the training set, in order, each one individually recorded. This is the block that makes the dataset reproducible and it is the one most often reduced to a single line saying “cleaned and normalized”.

Two details make this block defensible rather than decorative. First, records_in and records_out must reconcile across the whole chain, from source extract counts to final dataset count. An unexplained difference of forty rows is the kind of thing an inspector finds in five minutes. Second, exclusion rationale should point at a rule written before the exclusion, not a justification written after it. Draft Annex 22 makes exactly this point about test data: cleaning or exclusion should be documented and fully justified, and pre-processing should be pre-specified with a rationale that it represents intended use conditions.³ The same discipline applied to training data is not required by the draft, and it is still the right design.

Block 4: Labeling

Covered in its own section below, because labeler qualification is the field pharma teams most often leave empty and the one that carries the most weight in a GxP argument.

Block 5: Consent and personal data

The withdrawal field is uncomfortable and it belongs in the record anyway. A trained model does not forget a record because the underlying row was deleted. Deciding in advance whether withdrawal triggers retraining, triggers a documented risk acceptance, or is handled by the fact that the training data was irreversibly aggregated is better than discovering the question during an audit.

The dataset-level header

Above the five blocks is a short header that ties the record to the model and to the decision it supports.

Pinning a Dataset Version So Someone Can Rebuild It

A provenance record that describes a dataset nobody can reconstitute is a description, not a record. Pinning means that two years from now, a person who was not there can produce a byte-identical dataset from the record alone, or can prove that the stored dataset is the one described.

Three mechanisms, used together

Content addressing. Hash the assembled dataset and make that hash part of its identity. Any change produces a new identity rather than a modified old one with no visible difference. This is what makes the Part 11 requirement that record changes not obscure previously recorded information² achievable for a dataset rather than only for a form field.

Immutable storage. The dataset itself lives in storage that does not permit overwrite, with retention set to match the retention obligation on the decisions the model touched. Retention design has its own considerations at end of life, which we address separately in the article on decommissioning a GxP AI system.

A machine-readable manifest. The provenance record should exist in a structured format, not only as a PDF. Two established options are worth knowing. W3C PROV-O, a W3C Recommendation since 30 April 2013, models provenance as entities, activities, and agents connected by relations such as was generated by, used, and was attributed to.⁶ That maps cleanly onto a data pipeline: the dataset is an entity, each transformation step is an activity, and the person or service account is an agent. Croissant, developed through MLCommons and published as a specification with an accompanying peer-reviewed paper, is a JSON-LD metadata format for machine learning datasets built on schema.org, organized into dataset metadata, resource, structure, and semantic layers, and adopted by several major dataset repositories.¹⁰¹¹

The narrative layer still matters

Structured metadata answers what. It does not answer why. The datasheet concept, introduced by Gebru and colleagues, proposes that every dataset be accompanied by a document covering motivation, composition, collection process, recommended uses, and related considerations, on the model of a component datasheet in electronics.⁷ That narrative layer is what a reviewer reads first and what an inspector will ask about. The practical arrangement is a two-part record: a generated structured manifest that carries the facts, and a short approved narrative that carries the reasoning, with the narrative under document control and the manifest under record control.

Labeling, Labelers, and the Qualification Question

In a GxP setting the label is a judgment. When a person marks a vial image as a reject, that is a quality decision, and the usual rules about who is competent to make quality decisions do not stop applying because the decision happened during dataset preparation. Yet labeling is routinely treated as data preparation work, outsourced without qualification records, and documented nowhere.

Draft Annex 22 is clear on the test side: labeling of test data should be verified following a process that ensures a very high degree of correctness, which may include independent verification by multiple experts, validated equipment, or laboratory tests.³ For training labels the draft is silent, which leaves the design to the company. The defensible position is that training labels get a proportionate version of the same treatment, scaled to risk.

The labeling block

Two of these deserve emphasis. Agreement metric gives you a number to defend when someone asks how good the labels are. Without an overlap sample, there is no answer other than an opinion. Label change log matters because relabeling is common and rarely recorded. If a model underperformed and someone went back and corrected labels, that is a change to a GxP-relevant record, and Part 11’s requirement that changes not obscure previously recorded information applies to it.²

Rights, Licenses, and Consent Status

The rights block is the part of the provenance record most likely to be dismissed as a legal matter that data teams should not have to handle. It is also the part most likely to produce an expensive surprise, because rights change after collection while the dataset does not.

Rights drift is a real failure mode

The web domain audit cited earlier documented how quickly permission conditions can move: within roughly a year, the share of one widely used corpus restricted by terms of service reached 45 percent, and more than 28 percent of the most actively maintained sources in it became fully restricted through robots.txt.⁹ Public web data is the extreme case, but the same pattern shows up internally. A data sharing agreement with a contract manufacturer expires. An acquired company’s data turns out to have been collected under consent language that does not cover model development. A vendor changes its terms at renewal and the new terms restrict training use.

The provenance record handles this with two fields that most designs omit: expiry_or_review_date and a periodic reconfirmation step. Rights are given a review date the same way a supplier qualification is. When the date arrives, someone confirms that the basis still holds and records the confirmation. When it does not hold, that becomes a documented decision about the model, not a discovery two years later.

Borrowing the Commission’s structure

The Commission’s training content template for general-purpose AI models is a useful structural reference even for teams entirely outside its scope. Its source category listing (public datasets, licensed datasets, crawled or scraped online content, user data, synthetic data, other sources) is a reasonable taxonomy for classifying internal training sets, and its processing and governance section shows the kind of narrative a regulator considers adequate for copyright measures and data protection compliance.¹² Using a published regulatory structure rather than inventing one has an argumentative advantage: when an inspector asks why the record is organized this way, the answer points at a document the regulator wrote.

Keeping the Record Alive Through Retraining

A provenance record that is accurate on the day of first validation and stale six months later is worse than no record, because it invites reliance it cannot support. Retraining is where the record either holds or decays without anyone noticing.

The change control question around retraining, which decisions require what level of assessment and when a retrained model needs revalidation, is a topic in its own right and we treat it separately. What matters here is narrower: what happens to the provenance record when the dataset changes.

Version chains get long. A model retrained quarterly for three years has twelve dataset versions, and an inspector will not read all of them. What they will do is pick one, usually the one behind a decision they are already looking at, and follow it. The record has to be equally good at version seven as at version one, which is another argument for generating it rather than writing it.

Presenting the Record to an Inspector

Everything above is design. This section is delivery, and delivery is where good records still fail. An inspector has limited time and is not going to learn your data architecture. What they will do is ask a small number of questions and watch how quickly and confidently you answer.

The four questions to rehearse

Two further questions come up often enough to prepare for. “Which records did you exclude and why” goes straight to the transformation block and the reconciliation of record counts. “Was any of this data used to test the model as well as train it” goes to the partition map and the isolation controls, and it is the question that a poorly designed pipeline answers badly.

The walkthrough package

Prepare a short package per model, refreshed at each dataset version, rather than assembling one under pressure. In practice it contains four things: a one-page dataset summary showing sources, counts, date ranges, and the fitness statement; the full structured provenance record in human readable and electronic form, which is the Part 11 11.10(b) expectation applied literally²; the labeling summary with qualification references; and the version chain showing every dataset version behind every deployed model version.

Keep the raw manifest available but do not lead with it. Handing an inspector a JSON-LD file at minute three is a way of appearing to comply while making the reviewer’s job harder, and reviewers notice.

Where Provenance Records Fail

Five patterns account for most of the failures worth naming.

Reconstruction after the fact

The most common one. A model is already in use, provenance is requested, and someone spends six weeks rebuilding a history from Slack messages, notebook comments, and memory. The output is a plausible document that nobody can verify, which is a worse position than an honest gap, because it puts a signature on something unverified. The better response to a legacy model with no record is a documented gap assessment: state what is known, state what cannot be reconstructed, assess the risk, and decide whether the model continues, is retrained on a documented dataset, or is retired.

The vendor boundary

A model arrives trained. The vendor treats its training data as confidential. The regulated user still owns the obligation: draft Annex 22 says documentation should be available and reviewed by the regulated user whether the model was trained in-house or supplied by a vendor.³ Confidentiality is negotiable through structure. A vendor can usually provide source categories, date ranges, geographic coverage, labeling methodology and labeler qualification standards, and an attestation on rights and consent, without disclosing the data itself. Where a vendor will not provide even that, the record should say so plainly and the risk assessment should carry it. Transparency findings across major model developers show that copyright status, data access, and data labor remain areas of persistent opacity,¹⁴ so this is a predictable negotiation rather than an unusual one, and it belongs in the contract rather than in a support ticket after go-live.

The dataset that has no owner

Training sets are often assembled by a project team that disbands. The dataset then belongs to nobody: no one reviews its rights expiry, no one maintains its storage, no one notices when the underlying source system is decommissioned. Naming a data owner in the record header, with the same seriousness applied to system ownership, is a small field that prevents a large class of problems.

Documenting the pipeline instead of the dataset

A well-documented pipeline is not a provenance record. The pipeline describes what the code does in general. The record describes what happened to one specific dataset on one specific day with one specific set of parameters. Teams with mature engineering practices sometimes make this substitution honestly, pointing at a repository and a design document, and it does not answer the second inspector question at all.

Treating provenance as a compliance artifact only

The last one is a framing failure rather than a technical one. Teams that build the record purely to satisfy an anticipated inspection build it late, build it thin, and resent it. Teams that build it because it makes their own work possible get a different result. When a model underperforms at one site, the provenance record tells you whether that site was represented in training. When a class definition changes, the record tells you which datasets used the old definition. When a source system is upgraded, the record tells you which models depend on data from the prior version. That is engineering value, and it is the reason the record gets maintained between audits rather than only before them.

Conclusion

The regulatory position on training data provenance in September 2026 is genuinely unusual. The EU AI Act’s data governance duties for high-risk systems are deferred to December 2027 and August 2028. Draft Annex 22 has no final text and no implementation date, and what it does say concentrates on test data rather than training set origin. Part 11 tells you how to control a record without telling you what belongs in it. The GAMP AI guide and the EMA reflection paper give principles rather than schemas. There is, right now, no authority that will hand a pharma or biotech team a compliant field list.

That absence is the opportunity. The companies that design their provenance record now, while the requirement is a principle rather than a checklist, get to design it around how their data actually moves, test it against real inspector questions, and have two or three dataset versions of practice behind them before anything becomes binding. The companies that wait will be retrofitting a record onto models already in production, which is the expensive version of the same work and produces a weaker document. The fields in this article are a starting schema, not a standard. What makes any schema defensible is that it was designed deliberately, generated automatically, controlled properly, and rehearsed against the questions someone will actually ask.

Sakara Digital works with pharma and biotech organizations designing the records that have to hold up when an AI model touches a GxP decision. If you are building a training data provenance record and want an independent view on the schema, the controls around it, and how it will read to an inspector, we are happy to have that conversation.

For Further Reading