Human Review Sampling Plans for AI Outputs in GxP Work

A Reviewer Signature Is Not a Control

Walk through the AI inventory at most life sciences companies and a pattern shows up quickly. A model classifies deviations, drafts a batch record review summary, pulls adverse event terms from case narratives, or proposes a first draft of an annual product quality review section. In each case the risk assessment reaches the same mitigation: a human reviews the output before it is used. The mitigation is right. The problem is that it usually stops there.

Ask the follow-up questions and the answers are often missing. Does the reviewer look at every output or some of them? If some, which ones, and how were they chosen? What counts as an error? What error rate would be a problem? Who looks at the numbers, and how often? What changes if the numbers get worse? A review step that cannot answer these questions has no measured performance, which means nobody can show that it works.

That matters because the draft EU GMP Annex 22 on artificial intelligence, released for consultation in 2025 and still not final, treats the human reviewer as part of the controlled process. Where a model gives input to a decision made by a human operator and the testing effort for the model has been reduced, the draft says the intended use should include the operator’s responsibility, and that “the training and consistent performance of the operator should be monitored like any other manual process.”¹ In its operation section, the same draft says records should be kept from the human review, and that depending on the criticality of the process and the level of model testing, this “may imply a consistent review and/or test of every output from the model, according to a procedure.”¹ Annex 22 is a draft with no implementation date, so these are not requirements today. They are a clear signal of what an inspector will ask about.

Human Reviewers Miss Errors, and the Research Is Consistent About It

The case for measuring the review step rests on a long body of research. A 2015 study from Sandia National Laboratories on visual inspection of precision parts summarizes the prior literature this way: across most inspection tasks, missed defects typically fall between 20% and 30%.² In the study itself, trained inspectors correctly rejected 85% of defective items and incorrectly rejected 35% of acceptable parts.² These were careful people doing work with serious consequences, and they still missed defects.

Reviewing AI output adds a second problem on top of ordinary inspection error: the pull of a suggested answer. A 2021 experiment published in npj Digital Medicine gave 138 radiologists and 127 internal or emergency medicine physicians a set of chest X-ray cases with diagnostic advice, two of which carried inaccurate advice. Some advice was labeled as coming from an AI system and some from an experienced radiologist. Diagnostic accuracy was significantly worse when participants received inaccurate advice, whatever its stated source.³ The authors report that 27.54% of radiologists and 41.73% of the other physicians gave the wrong diagnosis every time they received inaccurate advice, while only 28.26% of radiologists and 17.32% of the other physicians rejected all the incorrect advice they were given.³ Expertise helped, but it did not protect most reviewers from a confident wrong suggestion.

A systematic review of automation bias in JAMIA adds a finding that matters for document-heavy GxP work. The authors found automation bias in single tasks, not only in multitasking settings, and linked it to tasks with high verification complexity, meaning tasks where checking the system’s answer takes real effort.⁴ Checking whether an AI-drafted investigation summary matches the underlying records is exactly that kind of task.

Two Error Rates, Not One

The practical consequence is that every AI use case with human review has two error rates. The first is the model’s: how often an output contains an error before anyone looks at it. The second is the reviewer’s: how often an error in the output gets past the review. What reaches the GxP record is the combination of the two.

A simple illustration shows why both matter. Suppose an AI tool produces a draft with at least one material error in 4% of outputs, and reviewers catch 75% of those errors. Then about 1% of approved outputs still carry a material error. Nobody would know that number unless both rates were measured. If the procedure only says “a qualified person reviews each output,” the company is claiming a 0% residual error rate without evidence, and a single inspector finding can disprove it.

Human review sampling AI outputs by risk, with the reviewer measured alongside the model, is how you replace that claim with a number you can defend. The rest of this article sets out how to build it.

When 100% Review Is the Right Answer

Sampling is not always appropriate. There are situations where every output should be reviewed before use, and a sampling plan should start by naming them. The NIST/SEMATECH engineering statistics handbook describes acceptance sampling as “the middle of the road” approach between no inspection and 100% inspection, and lists the usual reasons to sample instead of inspecting everything: testing is destructive, full inspection is too expensive, or it takes too long.⁶ None of those reasons applies automatically to AI review. Reviewing an output is not destructive, and for low volumes the time saved by sampling is small. So the default for anything important should be 100% review, and a use case should move to sampling only when its own data supports it.

Six Situations That Call for 100% Review

1. The output is the direct basis for a GxP decision with no independent later check. If an AI classification decides whether a deviation is escalated, and nothing downstream would catch a wrong call, every output needs a human decision.
2. The error rate is not yet known. A new model, a new prompt, a new intended use, or a new input population all mean you have no evidence about performance in this context. Start at 100% and collect the data that later justifies sampling.
3. The output comes from a generative model used anywhere near GMP decisions. The draft Annex 22 says generative AI and large language models should not be used in critical GMP applications, and that where they are used in non-critical applications, qualified and trained personnel “should always be responsible for ensuring that the outputs from such models are suitable for the intended use.”¹ Read plainly, that means a qualified person stands behind each output that is used.
4. The model flags low confidence. The draft Annex 22 expects classification models to log confidence scores and to consider flagging very low confidence outcomes as “undecided” rather than forcing an answer.¹ Undecided or low confidence outputs should route to full human review whatever the sampling rate is for the rest.
5. The volume is small. If a use case produces 20 outputs a month, a statistically meaningful sample is close to the whole population. Review everything.
6. The error type would be invisible later. Omissions in a regulatory summary or a wrong reference in a CAPA rationale may never surface through normal operations. If no later process would expose the error, review is the one step you cannot reduce.

100% Review Is Not 100% Effective

Choosing 100% review does not end the design work. Given the research above, a reviewer who sees every output will still miss some errors. Manufacturing has dealt with this for decades, and FDA’s approach to visual inspection of injectable products is a useful model.

FDA’s draft guidance on visible particulates in injectable products, issued in December 2021 and still a draft, calls for trained personnel, automated inspection, or both to inspect every unit, and then adds that “the quality unit should sample each batch for acceptance quality limit (AQL) testing.”⁵ It also says that following 100% inspection, manufacturers should use statistically sound sampling plans and appropriate acceptance criteria.⁵ In other words, the 100% inspection is the primary control and a separate statistical sample checks whether the primary control is working.

The same guidance gives a second useful principle: extrinsic particulates found during 100% inspection or AQL testing may mean the product is adulterated even when the statistical acceptance criteria are met.⁵ The AI equivalent is that some errors matter as single events, whatever the rate. A fabricated citation in a regulatory response or an invented lot number in a batch summary is a finding to investigate on its own, not just a data point in a trend.

Setting a Sampling Rate by Risk

Once a use case has earned sampling, the most common mistake is to pick a percentage. “We review 10% of outputs” sounds reasonable and is easy to write into an SOP. It is also the wrong unit. The confidence a sample gives you depends mainly on the number of outputs reviewed and the number of errors found, not on what share of the population they represent. Ten percent of 5,000 outputs a month is 500 reviews, which is far more than most tolerable error rates need. Ten percent of 80 outputs is 8 reviews, which proves almost nothing.

The better question has two parts. What error rate can this use case tolerate, and how confident do we need to be that the true rate is below it?

Start With a Tolerable Error Rate

Financial auditors have used this concept for decades. The PCAOB auditing standard on audit sampling defines the tolerable rate as the maximum rate of deviations from a prescribed control that the auditor would accept without changing the planned reliance on that control.⁷ The same standard ties sample size to three inputs: the tolerable rate, the expected rate of deviations, and the acceptable risk of wrongly concluding the control works.⁷

For AI review in GxP work, the tolerable rate should be set per error class, not as one number for the use case. A tolerable rate for critical errors might be effectively zero, which pushes that class to 100% review. A tolerable rate for minor formatting errors might be several percent. The draft Annex 22 offers an anchor for setting these values: it says a model’s acceptance criteria should be at least as high as the performance of the process it replaces, which means the performance of the current process has to be known.¹ If your current manual process for coding deviations has a known misclassification rate from past audits, that is a defensible starting point for the tolerable rate of the AI-assisted process.

Setting the tolerable rate is a quality decision, not a data science decision. The draft Annex 22 assigns the definition of acceptance criteria to a process subject matter expert, documented and approved before acceptance testing begins.¹ The same logic applies to the tolerable rate in the review plan.

Then Size the Sample From the Math

The simplest useful plan is a zero-acceptance plan: review a sample, and if you find no errors of the class in question, you can state an upper bound on the error rate at a chosen confidence level. If you find any, the plan fails and you investigate. The sample size comes from solving a simple binomial equation. There is a well-known shortcut for it: Hanley and Lippman-Hand, writing in JAMA in 1983, set out a quick rule for the maximum risk that is consistent with seeing no events in a sample of a given size.⁸ The rule, usually called the rule of three, says that if you observe zero events in n trials, the upper 95% confidence bound on the true rate is about 3/n. Zero errors in 300 reviewed outputs supports a claim of about 1% or less. Zero in 100 supports about 3% or less.

The table below gives exact sample sizes for common tolerable rates. The “c = 1” columns show the sample needed if the plan allows one error and still passes, which some teams prefer because a single minor slip does not force an investigation. These figures come from the binomial distribution and are the same numbers any statistician would calculate.

Two things stand out. First, the sample size for tight tolerable rates grows fast. Showing a 1% rate with 95% confidence takes about 300 clean reviews per period. For many AI use cases that is more than the monthly volume, which is another way of saying those use cases should stay at 100% review. Second, a small sample cannot show much. A sample of 59 clean outputs supports a 5% claim, not a 1% claim, no matter how carefully the reviewer worked.

A Three-Tier Structure for Review Intensity

Most companies do not need a custom plan for every use case. A small number of tiers, each with a defined review approach, covers most of the inventory and makes the plan easier to defend. The tier comes from the risk assessment already done for the use case.

Select the Sample Properly

A sample only tells you about the population if every output had a chance of being selected. The PCAOB standard puts it plainly: sample items should be selected so the sample can be expected to be representative, and “all items in the population should have an opportunity to be selected.”⁷ In practice this rules out letting reviewers pick the outputs they check, reviewing only outputs that someone complained about, or always taking the first outputs of the week.

Random selection should be done by the system or by a documented method, not by hand. Stratification adds a second layer. The draft Annex 22 asks that test data be stratified and include all subgroups of the intended use, with each subgroup large enough for adequate statistical confidence.¹ The same thinking applies to review sampling. If a model handles deviations from three sites, or documents in two languages, or both short and long narratives, a single random sample may contain very few of the rarer types. Sample within each stratum, and consider oversampling the strata where errors are most likely: low confidence outputs, rare input types, new sites, and new product families.

Using Accepted Sampling Standards Without Misusing Them

Quality professionals use familiar tools, and the most familiar sampling tool in pharma is the AQL table. That instinct is sound, because these standards encode decades of practical experience. It is also where a lot of misuse starts, so it helps to be exact about what the standards do and do not provide.

What the Standards Are

ANSI/ASQ Z1.4 and ASTM E2234 are the attribute sampling standards most quality teams know. ASTM describes E2234 as establishing lot or batch sampling plans for inspection by attributes, using MIL-STD-105E as a basis for sampling a steady stream of lots indexed by AQL.⁹ FDA’s visible particulates draft guidance notes that ASTM E2234 is equivalent to the ANSI/ASQ Z1.4 standards referenced in the USP chapters on visible particulates.⁵ ASTM’s list of applications is broad and explicitly includes “data or records” and administrative procedures, alongside end items and raw materials.⁹ So there is nothing strange about using these plans for AI outputs, as long as their assumptions hold.

The NIST handbook describes the foundation of the military standard as the AQL, and notes that the standard is not a single plan but a system: a normal plan, a tightened plan, and a reduced plan, plus rules for switching among them.¹⁰ That structure is the most useful part of these standards for AI review, and it is covered in detail later in this article.

What AQL Means, and What It Does Not

FDA’s draft guidance quotes the ASTM definition of the acceptance quality limit as the “quality limit that is the worst tolerable process average when a continuing series of lots is submitted for acceptance sampling.”⁵ Three parts of that definition matter for AI review.

• It describes a process average, not a single lot. An AQL plan is designed to accept lots at or better than the AQL most of the time. It does not promise that any single accepted lot is at or better than the AQL.
• It assumes a continuing series of lots. ASTM says the plans are intended primarily for a continuing series of lots and cautions that users applying them to isolated lots should consult the operating characteristic curves to find a plan with the protection they need.⁹ A one-off AI-assisted data migration is an isolated lot.
• It is framed around the producer’s risk. The NIST handbook explains that the producer designs plans with a high probability of accepting lots at the AQL, while the consumer protects itself through the lot tolerance percent defective, or LTPD.¹⁰ In AI review, the patient and the GxP record are the consumer. Your plan should be designed around the worst rate you can accept, not the rate you expect.

Five Ways Teams Misuse AQL Tables for AI Review

Translating Manufacturing Terms to AI Review

The FDA draft guidance lists what a firm should quantify when designing a sampling plan: operating characteristic curves for each defect class, accept and reject criteria, the AQL, and the unacceptable quality limit.⁵ It also says the method and acceptance criteria should consider patient risk, and that firms should select sampling plans according to the risk of a particular type of defect.⁵ The same guidance points out that CGMP regulations at 21 CFR 211.165 require batches to meet appropriate specifications and statistical quality control criteria as a condition for release.⁵ The mapping below shows how each concept carries over.

Define and Classify Errors Before You Count Them

An error rate is only as good as the definition of an error. Two reviewers looking at the same AI-drafted summary can reach different conclusions about whether a vague sentence is an error, a style issue, or fine. If the definition is loose, the error rate will move with reviewer mood and staffing, and trend charts will show noise instead of performance.

Choose the Counting Unit

The first decision is what you count. For a classification model, the unit is usually the output: the classification was right or wrong. For generative outputs, the choice is harder. Counting at the level of whole documents hides how many errors each document contains. Counting at the level of sentences or claims gives more sensitive data but takes more review effort.

A 2025 study in npj Digital Medicine shows what sentence-level counting looks like in practice. The authors, who were employees of the company developing the tool, evaluated LLM-generated clinical notes across 18 experimental configurations. Clinicians checked 12,999 note sentences against the source consultation transcripts and found a hallucination rate of 1.47%. They also checked 49,590 transcript sentences to see whether clinically relevant content was missing from the note, and found an omission rate of 3.45%.¹² This is a clinical documentation setting, not GxP work, but the design choices carry over directly.

Separate Wrong Content From Missing Content

The same study draws a line that most GxP review procedures do not: between content that is present but wrong (hallucinations) and content that should be present but is missing (omissions). The two call for different review methods. A reviewer can catch a wrong statement by checking each sentence against the source. Catching an omission requires reading the source and asking whether the output reflects everything that matters, which is slower and harder. Procedures that only ask reviewers to confirm what the output says will undercount omissions.

Classify by Consequence

The study’s severity rule is a good model. A hallucination or omission was labeled major if, left uncorrected, it could change the diagnosis or management of the patient, and minor otherwise.¹² By that rule, 44% of the hallucinated sentences were major, compared with 16.7% of the omissions.¹² Where two clinician reviewers disagreed, a senior clinician resolved the difference.¹² Adapted to GxP work, a three-class scheme usually works:

Write Examples, Not Just Definitions

Definitions alone do not produce consistent classification. Build a short library of real, anonymized examples from your own use cases: this is a critical error, this is major, this is minor, this is not an error. Update it whenever adjudication settles a borderline case. The library also becomes the raw material for reviewer qualification, covered in the next section.

Tracking Error Rates and Reviewer Performance

Collecting error data only helps if someone looks at it in a way that separates real change from normal variation. Three tools do most of the work: confidence intervals, control charts, and checks on the reviewers themselves.

Report an Interval, Not Just a Rate

An error rate from a sample is an estimate, and small samples give wide ranges. Reporting “2% error rate this month” without a range invites overreaction to noise. The NIST handbook recommends the Wilson method for confidence intervals on a proportion, noting that it holds up across different sample sizes and cannot produce a negative lower limit, and points to exact binomial methods for small samples or small defect counts.¹³

An example shows why the interval matters. Four errors in 200 reviewed outputs is a 2.0% observed rate. The 95% Wilson interval runs from about 0.8% to about 5.0%. If the tolerable rate is 3%, this sample has not shown the process meets it, even though the point estimate looks fine. Zero errors in 150 reviews gives a one-sided 95% upper bound of about 2.0%, which would support a 3% tolerable rate. The same arithmetic tells you how many reviews you need before the interval is narrow enough to act on.

Use a Control Chart for the Trend

For use cases in steady operation, a p-chart is the standard tool for tracking the proportion of nonconforming items over time. The NIST handbook gives the control limits as the center line plus or minus three times the square root of p(1−p)/n, where p is the baseline proportion and n is the sample size, and notes that the chart assumes a stable process with independent units.¹⁴

The formula also shows the limits of charting small samples. With a baseline error rate of 2% and 100 reviews per period, the upper control limit is about 6.2%, so the error rate would need to triple before the chart signals. With 400 reviews per period, the upper limit drops to about 4.1%. Low-volume use cases will not get useful signals from a p-chart alone, which is one more reason to pair the chart with event rules: any critical error triggers an investigation regardless of the chart, and two consecutive periods above the tolerable rate trigger tightened review.

Measure the Reviewer, Not Just the Model

The draft Annex 22 expects operator performance in human-in-the-loop use to be monitored like any other manual process.¹ Manufacturing again offers a proven method. For visual inspection, FDA’s draft guidance says personnel qualification should use a mixture of good units and defective units, that the quality unit should control the test sets so that qualification tests are not manipulated or biased, and that protocols should define requalification methods and frequency.⁵ It also calls for feasibility studies that cover a fatigue time frame, and for procedures that set a maximum inspection period without a rest break.⁵

Each of these translates to AI review:

• Seeded outputs. Quality prepares a controlled set of AI outputs with known, documented errors of each class. Reviewers are qualified against that set before they review live outputs, and requalified on a schedule. Some teams also mix a small number of seeded outputs into the live review queue to measure the real-world catch rate. If you do this, control the seeded set tightly and make sure seeded outputs can never reach a GxP record.
• Second-line agreement. The second-line re-review described earlier gives a direct measure of what first-line reviewers miss. Track it per reviewer, not only in total.
• Edit and override rate. If reviewers almost never change AI outputs, either the model is very good or the review is not happening. The second-line check and the seeded set tell you which.
• Time per review. A falling review time combined with a near-zero edit rate is an early sign that outputs are being approved without real checking. Compare review time to output length and complexity.
• Session length and workload. Set a maximum continuous review period and a daily volume limit per reviewer, based on the complexity of the outputs. The automation bias research links over-reliance to cognitive load and to tasks where checking the answer takes real effort.⁴

Design the Review Screen to Reduce Automation Bias

How the output is presented also affects how well it is reviewed. The chest X-ray experiment found that even specialists often followed inaccurate advice.³ Practical design choices that help include showing the source record beside the AI output rather than the output alone, requiring the reviewer to mark each claim as verified rather than approving the whole output with one click, and showing low confidence scores prominently. None of these removes the need to measure reviewer performance. They make the measured performance better.

When to Tighten, Relax, or Stop Review

A sampling plan that never changes is either too heavy for a well-performing use case or too light for one that is getting worse. The acceptance sampling standards solved this with written switching rules, and those rules adapt well to AI review.

How the Military Standard Switches

MIL-STD-105E, the basis for ASTM E2234 (which FDA describes as equivalent to ANSI/ASQ Z1.4), starts inspection at the normal level and moves between levels by fixed rules.¹¹

• Normal to tightened: when 2 out of 2, 3, 4, or 5 consecutive lots have been rejected on original inspection.¹¹
• Tightened to normal: when 5 consecutive lots have been accepted on original inspection.¹¹
• Normal to reduced: only when all of several conditions are met, including 10 preceding lots on normal inspection all accepted, a total defect count in those samples at or below a table limit, production at a steady rate, and reduced inspection considered desirable.¹¹
• Reduced to normal: when a lot is rejected, when production becomes irregular or delayed, or when other conditions warrant.¹¹
• Discontinuation: if lots continue to be rejected while on tightened inspection, inspection under the standard stops until corrective action is taken.¹¹

Two features of this design are worth copying. Moving to reduced inspection is deliberately harder than getting to tightened inspection. And the standard applies the switching rules to each class of defects independently, so a use case can be on reduced review for minor errors while staying on normal or tightened review for major ones.¹¹

Switching Rules Adapted for AI Review

The adapted rules below are a starting point, not a standard. Set the specific numbers in your procedure based on volume and risk, and justify them.

Change Control Sends the Use Case Back to Full Review

AI systems change in ways manufacturing processes usually do not: a vendor updates a model, someone edits a prompt, a new document template goes live. The draft Annex 22 says any change to the model, the system, or the process it supports should be documented and evaluated to decide whether the model needs retesting, and that a decision not to retest should be fully justified.¹ The review plan should say the same thing about review levels: a relevant change sends the use case back to entry-level review until new performance data exists. Evidence gathered on the old version does not describe the new one.

What Clinical Monitoring Teaches About Relaxing Review

Clinical development went through this debate over source data verification (SDV), the practice of checking trial data against source records. For years many sponsors checked close to everything. The industry group TransCelerate developed a risk-based monitoring method that moves away from a model rooted in SDV toward one built on cross-functional risk assessment and a mix of on-site, off-site, and central monitoring.¹⁵

Its 2014 analysis reviewed the literature, retrospective trial data, and member company audit findings, and concluded that the results support the view that generalized SDV has limited value as a quality control measure.¹⁵ The lesson for AI review is not that review is unimportant. It is that review effort spread evenly across everything is less effective than review focused on the outputs and error types that matter, with clear rules for when to increase it.

Writing the Sampling Plan Into Your Procedures

An inspector will not assess your statistics in the abstract. They will ask to see the procedure, the records that show it was followed, and the evidence that someone acted on the results. The draft Annex 22 expects records from human review, and a procedure governing it.¹ Manufacturing practice already works this way: FDA’s visible particulates draft guidance expects written procedures for 100% inspection and statistical sampling, and evaluation of trends in reject data.⁵ The steps below describe what a complete AI review sampling plan contains.

Keep the Plan in Proportion

Not every use case needs a standalone document. Many companies write one procedure that defines the tiers, the error class framework, the statistical method, the reviewer qualification approach, and the switching rules, and then attach a one-page plan per use case with the specific values. That keeps the method consistent and makes it easy to show an inspector how any single use case fits the overall approach.

Ownership

Split ownership along the lines the draft Annex 22 already suggests. The process subject matter expert owns the error definitions and tolerable rates, because only they know which errors matter in the process. Quality approves the plan and owns the second-line check, the seeded qualification set, and the switching decisions. The system owner keeps the sampling and recording mechanics working. Data science supports the statistical design and investigates model-side causes when rates move. When everyone owns the review plan, nobody does, so name each role in the procedure.

Conclusion

Human review is the control that makes most GxP AI use cases acceptable today, and it is also the control that is least often designed. The research on inspection accuracy and automation bias gives a clear reason to stop treating a reviewer’s signature as proof that errors were caught. The good news is that the quality function already has the tools: tolerable rates, statistically sized samples, defect classes, qualified inspectors with controlled test sets, control charts, and switching rules. Applied to AI outputs, those tools turn human review sampling AI outputs by risk from a promise into a number that can be defended in an inspection and improved over time. Start at 100% review, measure both the model and the reviewer, and relax only when your own data, at a stated confidence, says you can.

Sakara Digital works with pharma and biotech organizations that are putting AI into quality, clinical, and regulatory work and need the controls around it to hold up. If you are designing human review for AI outputs and want an independent view on where to start, which use cases need 100% review, and how to size and document the rest, we are happy to have that conversation.

For Further Reading