Data Quality Checks for AI Training Sets in GxP Before Anything Is Trained

Why the Checks Come Before Training

A model learns whatever patterns its training data contains, including the wrong ones. If some of the “reject” labels in a visual inspection dataset are wrong, the model learns to reproduce some of those errors. If every image of one defect type came from a single inspection line, the model may learn the line’s lighting rather than the defect. None of this can be removed afterward without going back to the data and training again.

That is the practical reason to put the data checks first. There is also a validation reason. In a GxP setting, the test is supposed to show that the model is fit for its intended use. A test can only do that if the data behind the model and the data used to test it are both sound, and if the two are properly separated. When a data problem is discovered after testing, the usual outcome is not a small fix. It is a retrain, a retest, a deviation, and a conversation about why the first test result was accepted.

The Evidence That Data Problems Are Common

Research on how AI projects go wrong points in the same direction again and again. In a study of 53 AI practitioners working on high-stakes applications, Google researchers described “data cascades,” meaning data problems that compound and cause harm further downstream. They reported a 92% prevalence of data cascades and described them as often invisible and delayed, but usually avoidable.⁸

A systematic review in Nature Machine Intelligence looked at machine learning models built in 2020 to detect or predict COVID-19 from chest images. From 2,212 studies identified, 62 passed the quality screening for full review. The authors concluded that none of those models was of potential clinical use, because of methodological flaws or underlying biases. Many of the problems were data problems: duplicated images across public sources, uncertainty about whether cases were confirmed, and combined datasets that included the same source twice.⁹

None of these studies is about pharmaceutical manufacturing. They matter here because the failure modes are general. The same kinds of errors show up whenever a team builds a dataset from existing records, adds labels, splits it, and trains. A GMP environment has one advantage: most of the source data already lives in controlled systems with audit trails. It also has one disadvantage: the data was collected to run and document a process, not to train a model, so its gaps and biases are rarely obvious.

What “Before Anything Is Trained” Means in Practice

The pre-training review is a decision point. The team assembles a candidate data pool, runs the five checks described below, writes down what it found, and gets agreement from the process subject matter expert (SME) and quality assurance (QA) that the data is fit to use. Only then does model training begin. For a low-risk use, that review may take a few days. For a model that will make or support batch disposition decisions, it will take longer, and it should.

What Regulators Ask For, and Where

Before building a checklist, it helps to be exact about what the regulatory texts say, because one common misreading causes real trouble. Two documents matter most for pharma and biotech, and both are drafts.

Draft EU GMP Annex 22 Is About Test Data

The European Commission opened a stakeholder consultation on 7 July 2025 on three GMP documents: a revised Chapter 4, a revised Annex 11, and a new Annex 22 on artificial intelligence. The consultation closed on 7 October 2025.¹ As of this writing, Annex 22 remains a consultation draft. There is no final text and no date by which it applies.

The draft covers AI models used in critical GMP applications with a direct impact on patient safety, product quality, or data integrity. It applies to static models with deterministic outputs.² Its two data sections, section 5 (Test Data) and section 6 (Test Data Independency), are written about test data. Section 5 asks that test data be representative of the full sample space of the intended use, stratified across subgroups, large enough for adequate statistical confidence, and labeled through a process that gives a very high degree of correctness. It says pre-processing should be pre-specified, exclusions documented and justified, and that generating test data or labels with generative AI is not recommended.²

Section 6 contains the independence requirement. It asks for technical or procedural controls that ensure “data which will be used to test a model, is not used during development, training or validation of the model.” Where test data is split from a larger pool before training, the draft says it is essential that “employees involved in the development and training of the model have never had access to the test data.” It adds access control, audit trails on the test data, no copies outside the repository, a record of when and how many times test data was used, and a four-eyes rule where staff separation is impossible.²

Where Draft Annex 22 Does Touch Training Data

The draft is not entirely silent on training. Section 2.1 asks for close cooperation between process SMEs, QA, data scientists, IT, and consultants during algorithm selection, model training, validation, testing, and operation. Section 3.1 asks for a detailed description of the intended use, including a full characterization of the input data and its common and rare variations (the “input sample space”), approved by a process SME. Section 6.4 says physical objects used for the final test should not have been used to train or validate the model. Section 7.1 says the test should detect possible over- or underfitting to the training data.² These give the training data an indirect target: it should cover the same sample space the test will check. But the detailed data rules are for the test set.

The FDA Draft Guidance Covers Development Data

The FDA’s January 2025 draft guidance, Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products, is also still a draft. It is broader in its treatment of training data. It says data used to develop AI models should be fit for use, meaning relevant and “reliable (i.e., accurate, complete, and traceable).”³

The guidance groups training and tuning data together as “development data.” For the credibility assessment plan, and commensurate with model risk, it asks sponsors to describe how the development datasets were split, how the data was collected, processed, annotated, stored, and controlled, the rationale for choosing the dataset, and how labels were established. On one specific point it is direct: “Explain how labels or annotations were established.”³

On testing, the FDA text says: “Test data should be independent of the development data and should not be shown to the algorithm during training.” It asks sponsors to specify how that independence was achieved and gives examples, including data acquired “using different batches or products.”³ For a manufacturing model, that example is useful, as the leakage section below explains.

A Note on Terminology

The two documents use the word “validation” differently, which causes confusion in cross-functional teams. Draft Annex 22 uses “validation dataset” in the machine learning sense: data used during development to decide how best to train the model.² The FDA guidance calls this “tuning data” and says it does not use the word validation in this context.³ In a GxP organization, “validation” already means computerized system validation (CSV), the documented evidence that a system does what it is intended to do. Pick one term for the tuning set and define it in your SOP. The table below maps the requirements.

Context From the Device Side

Medical device regulators set out related principles earlier. In October 2021, the FDA, Health Canada, and the MHRA published ten guiding principles for Good Machine Learning Practice (GMLP).⁴ The International Medical Device Regulators Forum released a final document built on these principles in January 2025.⁵ Principle 3 asks that training and test datasets represent the intended population. Principle 4 is titled “Training Data Sets Are Independent of Test Sets” and says: “All potential sources of dependence, including patient, data acquisition, and site factors, are considered and addressed to assure independence.”⁴ These principles are written for devices, not medicines manufacturing, but the reasoning carries over directly and many pharma data science teams already know them.

The practical conclusion: in a pharma GMP setting today, your pre-training data checks rest on a mix of draft regulatory text, general data integrity expectations, and good practice. That is not a reason to skip them. It is a reason to write your own standard, justify it, and apply it consistently.

Check One: Is the Data Representative of the Intended Use?

Representativeness is the check most teams believe they have done and most often have not. It asks whether the data pool covers the conditions the model will meet in routine use, in the right proportions, including the rare cases that matter most.

Start From the Intended Use, Not From the Data You Have

The usual way to build a training set is to pull what is available: the last two years of inspection images, all deviations in the eQMS, every batch record from one site. That gives you a dataset shaped by history, not by the intended use. The better sequence is the one draft Annex 22 uses for test data: describe the intended use first, characterize the input sample space and its subgroups, then check whether the data covers it.²

Draft Annex 22 lists the kinds of subgroups it has in mind: the decision output (for example, accept or reject), process baseline characteristics such as site or equipment, characteristics of the material or product, and characteristics of the task, such as defect type and severity.² Even though that list appears in the context of test data, it is a good starting grid for the training pool too. A model cannot learn a subgroup it has never seen.

Build a Coverage Matrix

A coverage matrix is a simple table. Rows are the subgroups your SME defined. Columns show how many records exist in the pool for each, how many are in each class, the date range, and the source systems. For a visual inspection model, the rows might be product presentation, fill volume, container type, inspection line, camera station, and each defect category. For a model predicting dissolution from process data, they might be formulation, API lot supplier, equipment train, and campaign.

The matrix answers three questions quickly:

• Gaps. Which subgroups have no data or too little to learn from?
• Imbalance. Which subgroups are so dominant that the model will mostly learn them?
• Confounding. Which subgroup variables move together, so that the model could learn one as a stand-in for the other? If all cracked-vial images came from Line 2, the model may learn Line 2.

That last point is well documented. A 2018 study in PLOS Medicine trained a deep learning model to detect pneumonia on chest radiographs from several hospital systems. The researchers found that a model trained to identify the hospital system could do so with near-perfect accuracy, and that the pneumonia model performed worse at an external hospital than on data from the hospitals it was trained on. They concluded that the hospital and department could confound the predictions, since disease rates differed between them.¹⁰ In a plant, the equivalent is a model that learns the site, the line, the camera, or the season instead of the quality attribute.

Rare Events Need Deliberate Collection

In GMP data, the events that matter most are usually the rarest: critical defects, out-of-specification results, unusual process excursions. A dataset pulled from routine records will contain very few of them. You have three honest options: collect more examples on purpose (for example, a defect library prepared and characterized under a protocol), narrow the intended use so the model is not asked to decide on cases it cannot learn, or route those cases to a human by design. Draft Annex 22 section 9.2 already points toward the third option, suggesting that teams consider whether a model should flag a result as “undecided” when its confidence is very low.²

Be careful with synthetic data used to fill gaps. Draft Annex 22 says generating test data or labels with generative AI is not recommended and any use should be fully justified.² For training data the draft is silent, but the same logic applies: synthetic examples reflect the assumptions of whatever generated them. If you use them, label them as synthetic in the dataset, keep them out of the test pool, and record why.

Time Is a Subgroup Too

Processes change. Raw material suppliers change, equipment is replaced, methods are revised, and specifications tighten. A training set that spans a major change contains two processes. Mark the known change control events on the timeline of your data and decide, with the SME, whether older data still represents the intended use. The FDA draft guidance makes the general point that models developed on historical data may not perform as well if that data differs from what the model sees in use.³

Check Two: Are the Labels Right?

For a supervised model, the labels are the definition of correct. If the labels are wrong, the model learns wrong answers and the test measures agreement with wrong answers. Label quality deserves its own check because the errors are hard to see. A mislabeled record looks exactly like a correct one.

Know What Each Label Is Based On

The first question for every label is where it came from. In GMP data, labels usually come from one of four places, and they are not equally reliable.

Draft Annex 22 asks that test labels be verified through a process that ensures a very high degree of correctness, which may include independent verification by multiple experts, validated equipment, or laboratory tests.² For training labels, a very high bar across the whole set is often impractical. What you need instead is to know the label source for each record, measure how accurate each source is on a sample, and decide what error rate the use can tolerate.

Write the Label Definition Down

Many label errors are not mistakes by the person labeling. They come from unclear definitions. Is a small cosmetic scratch a defect or not? Is a deviation caused by a missed SOP step “human error” or “procedure inadequate”? If two experienced inspectors would disagree, the dataset will contain both answers. Before labeling starts, write a labeling guide that ties each label to the relevant specification, defect library, or SOP, with worked examples of borderline cases. Put it under document control. If the definition changes during the project, record which records were labeled under which version.

Measure Agreement and Look for Errors

Two practical checks catch most problems.

• Agreement sampling. Have a second qualified reviewer label a random sample, stratified by subgroup, without seeing the first label. Report the agreement rate by subgroup. Low agreement on one subgroup usually means the definition is unclear there.
• Error screening. Statistical methods can flag records whose labels look inconsistent with the rest of the data. Confident learning, described by Northcutt, Jiang, and Chuang in 2021, estimates which labels in a dataset are likely wrong, and has been used to find label errors in widely used public datasets.¹³ These methods do not decide which label is correct. They produce a short list of records for an expert to review.

Record the results. If the agreement rate or the error screen leads you to relabel records, keep the original label and the new one, with who changed it, when, and why. This is ordinary data integrity practice, and the FDA draft guidance’s request to explain how labels were established is much easier to answer with that history in hand.³

Check Three: Can You Trace Every Record to Its Source?

Provenance means being able to say where each record came from, how it was extracted, and what was done to it on the way into the dataset. SD has written separately about digital provenance for training data in depth, so this section focuses on the checks to run before training.

Trace the Extraction, Not Just the Source System

“The data came from the LIMS” is not provenance. A reviewer needs to know which system and version, which query or export produced the extract, the date and time of extraction, who ran it, the filters applied, and the file or table checksum of the result. If the extract was transformed (units converted, images cropped, fields merged), each step should be scripted, version-controlled, and repeatable. A manual spreadsheet step in the middle of a pipeline is where traceability usually breaks.

The FDA draft guidance treats traceability as part of reliability, alongside accuracy and completeness.³ In GMP terms, the question is the familiar one: can you reconstruct how this data came to be what it is?

Watch for Combined and Duplicated Sources

The COVID-19 imaging review found what its authors called “Frankenstein” datasets: public datasets assembled from other datasets and redistributed under a new name. In one case, a combined dataset included a component that already contained another of its components, so the same images appeared more than once without anyone noticing.⁹ Pharma teams face the same risk in a different form. Data may be pulled from a data lake that already merged two systems, from a vendor’s pre-built dataset, and from a site’s local archive, all of which may contain overlapping records.

Before training, check for exact and near-duplicate records across all sources. For structured data, match on batch number, sample ID, and timestamp. For images, compare file hashes for exact copies and use image similarity for near-copies, such as the same vial photographed twice or the same image saved at two resolutions. Duplicates inside the training set distort the weighting. Duplicates across training and test sets are leakage, covered in the next check.

Supplier Data and Supplier Models

If a vendor trained the model or supplied the dataset, provenance does not become the vendor’s problem. Draft Annex 22 section 2.2 says documentation of these activities should be available and reviewed by the regulated user whether the model is trained, validated, and tested in-house or provided by a supplier or service provider.² Ask for the dataset description, the label sources, the split method, and the duplicate checks as part of supplier qualification. If a supplier cannot describe its training data, that is a finding in itself.

Completeness Is Part of Provenance

An extract that dropped records without anyone noticing is a provenance problem. Reconcile record counts between the source system and the extract. Check what happened to records with missing fields, invalidated results, or reprocessed data. In chromatography, for example, decide with QA whether reprocessed results, invalidated runs, and system suitability failures belong in the pool, and document the rule. Excluding them may be right, but it has to be a documented decision, not a side effect of a query filter. Draft Annex 22 asks that any cleaning or exclusion of test data be documented and fully justified.² Applying the same discipline to the training pool takes little extra effort and prevents questions later.

Check Four: Is Anything Leaking Between Training and Test?

Leakage happens when information from the test data reaches the model during training, or when the test data is not a fair sample of the conditions the model will face. The model then looks better in testing than it will perform in use. Of the five checks, this is the one most closely tied to regulatory text, because both draft Annex 22 and the FDA draft guidance require test data to be kept independent of development data.²³

It belongs in a pre-training review because most leakage is created at the moment the data is split. Once the test set is defined and locked, it is hard to prove after the fact that nothing crossed over.

The Leakage Taxonomy

Sayash Kapoor and Arvind Narayanan surveyed published research using machine learning and found leakage errors affecting 329 papers across 17 fields. They grouped the errors into eight types.¹¹ The table below restates their categories and gives an example of how each can occur with GMP data.

Split by Group, Not by Record

The most common GMP leakage is non-independence. A batch produces many images, many in-process measurements, and many test results, and records from the same batch are more alike than records from different batches. If you split randomly by record, the model sees parts of a batch in training and other parts in testing, and the test measures how well it recognizes batches it already knows.

The fix is a group split: every record from a given batch (or lot, or unit, or campaign) goes entirely into training or entirely into test. Standard machine learning tools support this. The scikit-learn documentation, for example, explains that the usual independence assumption breaks when data comes in groups of dependent samples, and describes splitters that keep each group entirely within one fold.¹⁵ The FDA draft guidance’s example of achieving independence with data acquired from different batches or products describes the same idea at a regulatory level.³ GMLP principle 4 names patient, data acquisition, and site factors as sources of dependence to address.⁴

Decide the grouping variable with the SME. For a tablet coating model, the batch may be right. For a cell therapy process, it may be the donor. For a visual inspection model, it may be the individual container, since draft Annex 22 section 6.4 says physical objects used for the final test should not have been used to train or validate the model, unless features are independent.²

Duplicates Across the Split Inflate Results

Duplicates matter more than their numbers suggest. Researchers examining two widely used image benchmark datasets found that 3.3% of CIFAR-10 test images and 10% of CIFAR-100 test images had duplicates in the training set. When they replaced those images with new ones, the models they evaluated showed a relative drop in classification accuracy of between 9% and 14%.¹² A test set with hidden duplicates rewards memorization. Run duplicate detection on the full pool before the split, and again on the final training and test sets as a confirmation.

Lock the Test Set Before Development Starts

Draft Annex 22 describes two ways to protect test data: capture it only after training and validation are complete, or split it from the full pool before training starts.² For most manufacturing projects, the split-first approach is more practical, which makes the lock-down the final step of the pre-training review. The draft’s controls for this approach are specific: access control, an audit trail of access and changes, no copies outside the repository, a record of when and how often the data was used, and separation between staff who have seen the test data and those who train the model.²

Check Five: Is It Written Down Before Anyone Trains?

The first four checks produce findings. The fifth check makes sure those findings, and the decisions they led to, exist as an approved record before training starts. A pre-training review that lives only in a data scientist’s notebook will not help during an inspection, and it will not help the next team that retrains the model.

A Dataset Specification

The central document is a dataset specification. It describes the approved data pool and each subset (training, tuning, and test) in enough detail that someone else could rebuild it. The machine learning research community has proposed structured formats for this. Data Cards, proposed by researchers at Google, are structured summaries of the facts about a dataset that people across its lifecycle need, including upstream sources, collection and annotation methods, and the reasons behind them.¹⁴ In a GxP setting, the dataset specification serves that purpose but is kept under document control with defined approvers.

A practical dataset specification includes:

• The intended use and the input sample space it was built against, with a reference to the approved intended use description
• The source systems, extraction methods, dates, and record counts, with reconciliation to the source
• The coverage matrix, with known gaps and the decisions taken about them
• The label sources, labeling guide version, agreement results, and relabeling history
• Inclusion and exclusion rules, with justification for each
• Duplicate detection method and results
• The grouping variable and split method, with record counts in each subset by subgroup
• Pre-processing steps and where each one was fitted
• File identifiers or checksums for each frozen subset
• The access controls applied to the test set

Most of this is what the FDA draft guidance asks sponsors to describe for development data in a credibility assessment plan: how the data was split, collected, processed, annotated, stored, and controlled, why it was chosen, and how labels were established.³ Writing it before training means you are recording decisions, not reconstructing them.

Freeze the Versions

Once approved, the training, tuning, and test sets should be frozen: stored as fixed versions with identifiers, so the model can be linked to the exact data it was built and tested on. If the data has to change later, that is a new version with its own record. This matters when the model is retrained. A retrained model trained on “the updated dataset” is hard to assess if nobody can say what changed.

Who Approves

Draft Annex 22 gives the process SME responsibility for the intended use description and the acceptance criteria, both approved before acceptance testing begins, and calls for close cooperation between SMEs, QA, data scientists, and IT.² A reasonable approval set for the dataset specification is the process SME (for representativeness and label definitions), the data science lead (for the technical method), and QA (for data integrity and completeness of the record). For higher-risk uses, add the system owner.

Running the Pre-Training Data Review

The five checks work best as one structured review with a clear owner, a sequence, and an exit decision. The sequence matters because some steps depend on others. You cannot check representativeness without a defined intended use, and you should not explore the data before the test set is split and locked.

Label verification for the test set comes after the lock-down in this sequence on purpose. It means the people verifying test labels work inside the controlled repository and are recorded as having had access, which keeps the staff independence record clean. If your team is small, plan this early. Draft Annex 22 allows a four-eyes arrangement where full separation is impossible, but it has to be designed in, not claimed afterward.²

Scale the Review to the Risk

Draft Annex 22 section 2.3 says the activities it describes should be implemented based on risk to patient safety, product quality, and data integrity.² The FDA draft guidance repeatedly scales its requests to model risk.³ The same applies to a pre-training review. A model that suggests a deviation category for a human reviewer to confirm does not need the same depth as a model that rejects vials without human review. What should not change with risk is the order: the checks happen before training, even when they are light.

What to Do When a Check Fails

A failed check is a useful result, not a delay. The common outcomes are:

• Collect more data for a missing or thin subgroup, under a protocol.
• Narrow the intended use so the model is not asked to decide on conditions the data does not cover, and route those cases to people.
• Relabel after fixing the label definition, with the history retained.
• Re-split by the correct grouping variable, before anyone has trained on the old split.
• Stop. Sometimes the data cannot support the intended use. Finding that out before training is far less disruptive than finding it out after a failed test or, worse, in operation.

Who Tends to Be Missing

The people most often absent from data work are the process SMEs. Data scientists can run duplicate detection and group splits. They usually cannot tell you that two inspection stations had different camera firmware for six months, that a deviation category was redefined in 2024, or that a raw material supplier change altered a spectral signature. Draft Annex 22 names close cooperation between SMEs, QA, data scientists, and IT as a principle for exactly this reason.² ISPE’s AI Community of Practice has also listed a data quality framework for AI and machine learning, building on existing data integrity work, among its planned deliverables, which suggests industry guidance on this topic is still being written.⁷

Conclusion

The regulatory texts for AI in pharma manufacturing are still drafts, and the most detailed of them, draft EU GMP Annex 22, focuses its data requirements on test data. That is sensible: the test is where fitness for intended use is demonstrated. But a test only confirms what the data allowed the model to learn. Representativeness, labeling quality, provenance, leakage, and documentation are decided when the data pool is assembled and split, and that happens before anything is trained. Teams that treat these as pre-training checks spend less time retraining, write shorter deviation reports, and can answer an inspector’s questions about the data without reconstructing history.

Sakara Digital works with pharma and biotech organizations that are building their first GxP machine learning models and the data practices behind them. If you are planning a model for a quality or manufacturing use and want an independent view on whether your data is ready to train on, we are happy to have that conversation.

For Further Reading