Agentic AI in Quality Operations: Where Autonomy Is Acceptable and Where It Is Not

What Agentic AI Changes for a Quality Organization

Most quality teams met generative AI as a chat window. Someone pasted in a deviation description, asked for a cleaner summary, copied the answer back into the QMS, and signed it. The human did every step that touched a system of record. The AI only answered.

Agentic AI removes that separation. Anthropic’s engineering guidance draws the line clearly: workflows are systems where language models and tools follow predefined code paths, while agents are systems where the model directs its own process and tool use and keeps control over how it completes the task.¹ The same guidance notes that agents plan and operate on their own once a task is clear, and may return to a human for information or judgment. It also warns that autonomy brings the potential for compounding errors and recommends testing in sandboxed environments with appropriate guardrails.¹

In a quality organization, that means an agent could open a deviation record, query the manufacturing execution system for the batch history, pull environmental monitoring results, search five years of past investigations, draft a root cause hypothesis, propose a CAPA, route it for approval, and set the due dates. Each of those steps is technically possible today. Some of them are fine to hand over. Some of them are not, and the reason is not the quality of the model. It is who the regulations say must decide.

From Answering to Acting

The practical shift is from an AI that produces text a person then uses, to an AI that takes actions a person may or may not see before they take effect. That changes three things for quality leaders.

• The review point moves. With a chat tool, the human review happens naturally, because the person has to copy the output somewhere. With an agent, review happens only where the system forces it.
• The scope of access grows. An agent needs credentials to the QMS, LIMS, MES, and document management system to do useful work. Whatever those credentials allow, the agent can do.
• The error pattern changes. A single wrong answer in a chat is contained. A wrong intermediate result in an agent’s plan feeds every later step, which is exactly the compounding effect the Anthropic guidance describes.

ISPE’s Pharmaceutical Engineering reported in its July/August 2026 issue that several agentic AI tools built for root cause analysis, investigations, and CAPA were already demonstrated in 2026.² The same authors proposed a guardrail framework specifically to keep such tools from driving critical quality decisions on their own. The technology has arrived in the exact processes where the question of autonomy matters most.

Autonomy Is a Design Choice, Not a Side Effect of Capability

A useful idea comes from researchers at the University of Washington. Feng, McDonald, and Zhang argue that an agent’s level of autonomy can be treated as a deliberate design decision, separate from its capability and the environment it works in.³ They define five levels by the role the user plays: operator, collaborator, consultant, approver, and observer. At the approver level, the user steps in only when the agent hits a blocker, which includes signing off on consequential actions. At the observer level, the user can watch activity logs and pull an emergency off-switch, but cannot redirect the work.³

This framing matters because vendors often present autonomy as a feature that comes with a more capable model. It does not have to. A highly capable agent can be deployed at a low autonomy level, and in quality operations it often should be. The capability decides what the agent could do. The quality organization decides what it may do.

What the Rules Already Say About Who Decides

Before designing any oversight model, it helps to list the decisions that the regulations already assign to a specific person or unit. None of these rules mention AI agents. All of them constrain what an agent can be allowed to do.

The Quality Control Unit’s Authority Under 21 CFR 211

Under 21 CFR 211.22(a), the quality control unit has “the responsibility and authority to approve or reject all components, drug product containers, closures, in-process materials, packaging material, labeling, and drug products.”⁴ Paragraph (c) gives the same unit responsibility for approving or rejecting all procedures or specifications that affect the identity, strength, quality, and purity of the drug product, and paragraph (d) requires those responsibilities to be in writing.⁴

Section 211.192 adds that production and control records “shall be reviewed and approved by the quality control unit to determine compliance with all established, approved written procedures before a batch is released or distributed.”⁵ The same section requires that any unexplained discrepancy or specification failure be thoroughly investigated, and that the investigation extend to other batches that may be associated with it, with a written record including conclusions and follow-up.⁵ Section 211.165 requires laboratory determination of conformance to final specifications for each batch before release, and states that drug products failing to meet specifications shall be rejected.⁶

The words that matter here are “approve,” “reject,” and “review and approve.” They describe acts of judgment by a unit of people with written responsibilities. An agent can prepare the evidence for those acts. It cannot be the one performing them.

The Qualified Person Under EU GMP Annex 16

In the EU, Annex 16 states that each batch of finished product must be certified by a Qualified Person (QP) within the EU before release for sale, supply, or export.⁷ The annex also contains a distinction that is directly useful for agent design. Section 1.6 lists operational responsibilities the QP must personally ensure before certification. Section 1.7 lists a longer set of checks (points 1.7.1 to 1.7.21) that may be delegated to appropriately trained personnel or third parties, and it notes that the QP will need to rely on the pharmaceutical quality system and should have ongoing assurance that this reliance is well founded.⁷

Read with agents in mind, Annex 16 already separates preparation from certification. Many of the checks that feed a certification decision can be done by others under the quality system. The certification itself, and the personal responsibilities in 1.6, cannot be handed to anyone else. If an agent helps with the delegable checks, the QP’s reliance on it must be “well founded,” which in practice means validated, monitored, and documented.

Electronic Signatures Belong to Individuals

21 CFR Part 11 defines an electronic signature as a computer data compilation of symbols “executed, adopted, or authorized by an individual to be the legally binding equivalent of the individual’s handwritten signature.”⁸ Section 11.100(a) states: “Each electronic signature shall be unique to one individual and shall not be reused by, or reassigned to, anyone else.”⁹ Section 11.50 requires signed records to show the printed name of the signer, the date and time, and the meaning of the signature, “such as review, approval, responsibility, or authorship.”¹⁰

An agent is not an individual. It cannot hold an electronic signature, and a person cannot lend it theirs. Any design where an agent acts under a person’s credentials and applies that person’s signature, even with the person’s agreement, breaks the link Part 11 exists to protect. The practical rule is simple: an agent may author a draft, and the record can say so. It may never be the signer of a review or approval.

What Draft Annex 22 Adds, and What It Does Not

The EU’s draft Annex 22 on artificial intelligence, published for consultation in July 2025, is the most direct regulatory statement on this subject. It is still a draft. There is no final text and no implementation date as of this writing, so treat it as a clear signal of inspector thinking, not as a requirement in force.

The draft applies to AI models used in “critical applications with direct impact on patient safety, product quality or data integrity.”¹¹ It covers only static models with deterministic output. It says that dynamic models and models with probabilistic output should not be used in critical GMP applications, and then states that the document does not apply to generative AI and large language models, and “such models should not be used in critical GMP applications.”¹¹ For non-critical applications, the draft says “personnel with adequate qualification and training should always be responsible for ensuring that the outputs from such models are suitable for the intended use,” which it calls a human-in-the-loop.¹¹

Agents of the kind Anthropic describes are, by definition, built on large language models. Under the draft, that places them outside critical GMP applications and inside a human-in-the-loop model for everything else. A 2026 paper in the PDA Journal of Pharmaceutical Science and Technology interpreting the July 2025 draft describes it as defining expectations for intended use, validation, life cycle management, explainability, and human-in-the-loop oversight.¹²

The joint FDA and EMA Guiding Principles of Good AI Practice in Drug Development, released in January 2026, point the same way without setting specific rules. Principle 2 calls for proportionate validation, risk mitigation, and oversight based on the context of use and model risk. Principle 8 says “Risk-based performance assessments evaluate the complete system including human-AI interactions.”¹³ The principles cover manufacturing as well as nonclinical, clinical, and post-marketing uses.

A Decision Framework for Autonomy in Quality Operations

The regulations tell us where the fixed points are. They do not tell a quality leader what to do with the hundreds of smaller tasks between a deviation being logged and a batch being released. For that, we need a way to sort tasks.

Start With the Type of Work, Not the Process

A long-standing model from human factors research is a good place to start. In 2000, Parasuraman, Sheridan, and Wickens proposed that automation can be applied to four broad classes of functions: information acquisition, information analysis, decision and action selection, and action implementation. Each can be automated at a different level, from fully manual to fully automatic, and one system can mix levels across the four.¹⁴ They also observed that automation “does not merely supplant but changes human activity and can impose new coordination demands on the human operator.”¹⁴

This split is more useful than asking whether “deviation management” or “batch release” can be automated, because every quality process contains all four types of work. A deviation investigation includes gathering records (acquisition), comparing them to past events (analysis), choosing a root cause and a CAPA (decision), and updating the QMS record (implementation). The right level of autonomy differs across those four steps inside the same process.

Four Oversight Levels

With the type of work identified, each task can be placed at one of four oversight levels. We describe them by what the human does, following the user-role approach from Feng and colleagues, but tied to GxP accountability rather than general software use.

Note what is missing. There is no level where the agent acts on a GxP record and the human merely observes. In the Feng framework, that would be the observer level, where the user can watch logs and use an off-switch but cannot direct the work.³ For quality decisions under current regulations, we do not see a place for it. Observer-style autonomy can be acceptable for non-GxP administrative tasks, but those fall outside this framework.

The Four-Question Test

To place a task, ask four questions in order. The first “yes” that points to a higher level of control settles the answer.

1. Is the step assigned by regulation or by your quality procedures to a named person or unit? Release, certification, approval or rejection of product, approval of procedures and specifications, and signature all fall here. If yes: Level D.
2. Does the step create, change, or approve content in a GxP record? If yes, and it is not a Level D act, the agent may draft but a named person must review and own the content before it becomes part of the record: Level B at minimum.
3. Could a wrong output reach product or patient, or change a regulatory commitment, before an independent human check catches it? If yes, the agent may recommend but may not decide: Level C at minimum, and often Level D.
4. Is the step read-only, and would any error be caught by the human process it feeds? If yes, Level A is acceptable, provided the agent’s access is technically limited to reading.

One more filter runs across all four questions: the type of model. If the agent is built on a large language model, the draft Annex 22 position is that it should not be used in critical GMP applications, and that qualified personnel should be responsible for its outputs in non-critical ones.¹¹ Even though the annex is not final, a quality organization that places an LLM-based agent at Level A for anything with a direct effect on patient safety, product quality, or data integrity should expect to defend that choice at inspection.

Where Autonomy Is Acceptable

The good news for quality teams is that a large share of the work that slows investigations and reviews is information gathering and drafting. Those are the areas where agents can take on real work without crossing any of the fixed lines above.

Data Gathering and Evidence Assembly (Level A)

ISPE’s guardrail article describes the problem well: investigations require pulling data from the QMS, MES, LIMS, change control, training records, and SOPs, and quality professionals must query each system by hand, export data, and look for correlations, a process that leaves room for error and incomplete analysis.² An agent that assembles this evidence pack is doing information acquisition. If its access is read-only, it cannot change a record. If the investigator reviews the pack before relying on it, any gap or wrong pull is caught in the normal course of the investigation.

Typical Level A tasks in quality operations include:

• Retrieving the batch record, equipment logs, environmental monitoring results, and material certificates for a batch under investigation.
• Searching past deviations, investigations, and CAPAs for similar events by product, equipment, failure mode, or supplier.
• Compiling the document set for an Annual Product Quality Review or a periodic review.
• Checking that every deviation linked to a batch is closed before the batch reaches the release queue, and flagging any that are not.
• Sending reminders on CAPA due dates, overdue training, or open change control tasks.

The last two items show that Level A can include actions, as long as those actions do not change a GxP record or make a quality decision. A flag that says “two linked deviations are still open” starts a human process. It does not hold, release, or reject anything.

Drafting (Level B)

Drafting is where many quality teams see the first large time savings, and it is acceptable autonomy as long as the draft does not become the record without a person taking ownership. Good Level B candidates include:

• First drafts of deviation descriptions and immediate-action summaries from operator notes and system data.
• Investigation report sections that describe the timeline, the data reviewed, and the hypotheses considered.
• Proposed CAPA wording and effectiveness check criteria.
• Change control impact assessments that list affected documents, systems, validations, and training.
• Redlines of SOPs after a CAPA, prepared for the document owner’s review.
• Narrative sections of an Annual Product Quality Review, with the underlying data tables linked.

Level B works only if the review is substantive. The reviewer needs the agent’s sources alongside the draft, so they can check claims rather than read prose. ISPE’s framework makes the same point: every AI response in its model includes citations to source documents, the reasoning from data to conclusion, confidence scores, and alternative hypotheses when confidence is moderate.² A draft without its evidence invites approval by reading rather than approval by checking.

Monitoring and Early Signals (Level A Feeding Level C)

Agents can watch data streams and quality records continuously in a way no team can. Trending deviations by equipment, spotting a rise in a specific failure mode, or noticing that a supplier’s certificates changed format are all useful signals. The agent’s job is to raise the signal with its evidence. What happens next, whether to open an investigation, notify a supplier, or escalate to management, is a decision and belongs at Level C or D.

Where Autonomy Is Not Acceptable

The list of Level D tasks is shorter, and it should be treated as fixed unless the regulations change. These are acts, not analyses. An agent can contribute to the work that comes before them, but not to the act itself.

Batch Release and QP Certification

Release is the clearest case. Under 211.22 and 211.192, the quality control unit approves or rejects drug products and reviews and approves production and control records before a batch is released or distributed.⁴⁵ In the EU, the QP certifies each batch.⁷ An agent may run a pre-release completeness check and present an exceptions list. It should have no system permission to change a batch status to released, and no role that could be read as the release decision.

This holds even for “review by exception” designs, where the system presents only the anomalies for human review. Review by exception is a common approach for electronic batch records. What makes it acceptable is that the exception rules are defined, validated, and deterministic, and that a person still makes the release decision. An LLM-based agent deciding what counts as an exception is a different thing, and under the draft Annex 22 position it would be a probabilistic model used in a critical application.¹¹

Final Disposition of Product and Materials

Approving, rejecting, quarantining for reprocessing, or releasing materials and product are the quality unit’s decisions under 211.22(a).⁴ Section 211.165 states that drug products failing to meet specifications shall be rejected, and that reprocessed material must meet appropriate standards before acceptance.⁶ An agent can compile test results, compare them to specifications, and highlight out-of-specification values. It must not set the disposition.

Approval of GxP Records and Electronic Signatures

No agent should approve an investigation report, a CAPA plan, a change control, a validation deliverable, a batch record, or any other GxP record. Part 11 ties each electronic signature to one individual and requires the meaning of the signature to be shown.⁹¹⁰ An approval that was generated by an agent and applied under a person’s account is not that person’s approval, whatever the system shows.

The same applies to approving procedures and specifications, which 211.22(c) assigns to the quality control unit.⁴ An agent may redline an SOP. The document owner and quality approver sign it.

Final Root Cause and Investigation Closure

Section 211.192 requires discrepancies to be thoroughly investigated, the investigation extended to other potentially affected batches, and a written record made with conclusions and follow-up.⁵ An agent can propose hypotheses and even rank them. Selecting the root cause, deciding which other batches are affected, and closing the investigation are decisions. ISPE’s framework, even for its lower-severity categories, keeps a human reviewing findings before anything is approved, and for serious problems requires human signoff before any corrective action is taken.²

Regulatory Reportability

Some quality events trigger reporting obligations with short deadlines. Under 21 CFR 314.81, an NDA applicant must submit a Field Alert Report within three working days of receiving information about, for example, bacteriological contamination or a distributed batch failing its specification.¹⁵ An agent can and should help here, by flagging events that may meet the criteria and assembling the facts quickly. Whether the event is reportable, and what the report says, is a regulatory judgment. It stays with the people accountable for it.

The Gray Zone: Triage, Classification, and Closure Recommendations

The hardest cases are not release or drafting. They are the recommendations that come between them: classifying a deviation as minor, major, or critical; recommending that no CAPA is needed; proposing that an event is an isolated occurrence; suggesting an investigation can close. These are Level C tasks. The agent recommends and a named person decides. The difficulty is making sure the person’s decision is real.

Automation Bias Is the Main Risk

Automation bias is described in a 2012 systematic review by Goddard, Roudsari, and Wyatt as “the tendency to over-rely on automation.”¹⁶ The review screened 13,821 papers and included 74. It found that environmental factors, including workload, task complexity, and time constraint, mediated over-reliance by putting pressure on cognitive resources. It also found that mitigators included training, emphasizing user accountability, attaching updated confidence levels to the system’s output, and providing information rather than a recommendation.¹⁶

Every one of those mediators is present in a busy quality unit: high volume, complex events, and deadlines. That is the environment in which a reviewer is most likely to accept a triage recommendation without independent thought. A Level C design that ignores this becomes Level D in name and Level A in practice.

Design Choices That Keep the Human Decision Real

• Present evidence before the recommendation. Show the data the agent used, and let the reviewer form a view, before showing the agent’s classification. The Goddard review lists providing information rather than a recommendation as a mitigator.¹⁶
• Allow the agent to say “undecided.” The draft Annex 22 suggests that where a model’s confidence score is very low, it should be considered whether the model should flag the outcome as “undecided” rather than making a potentially unreliable classification.¹¹ An agent that always gives an answer trains reviewers to expect one.
• Route by severity, not by agent confidence alone. ISPE’s framework ties oversight to risk: in its worked example, a critical patient safety event went to the highest oversight category even though its calculated risk score pointed one level lower, and all AI hypotheses required expert validation before the investigation could proceed.²
• Record the human’s decision separately from the agent’s recommendation. The QMS should show what the agent proposed, what the person decided, and whether they agreed. That record is also your best evidence at inspection.
• Protect reviewer time. If throughput targets assume a two-minute review for an agent-classified deviation, the review will take two minutes whether or not the classification is right.

Monitor the Reviewer, Not Only the Agent

Draft Annex 22 contains a line that deserves more attention than it gets. Where a model gives input to a decision made by a human operator, and testing effort on the model has been reduced on that basis, “the training and consistent performance of the operator should be monitored like any other manual process.”¹¹ Section 10.5 adds that records should be kept of the human review, and depending on criticality, this may mean consistent review or test of every output.¹¹

In practice, that means measuring things like how often reviewers change the agent’s classification, how long reviews take, and whether override rates differ sharply between reviewers. An override rate near zero across hundreds of recommendations is not proof that the agent is excellent. It may be proof that nobody is checking. The FDA and EMA principles make the same point from the other direction: performance assessment should cover the complete system, including human-AI interactions.¹³

Build the Boundaries Into the System, Not the Prompt

The most common design mistake we see in early agent projects is enforcing the autonomy level through the agent’s instructions. A system prompt that says “never approve a record” is not a GxP control. The model may follow it most of the time. A control is something that works every time, whether or not the model behaves.

Excessive Agency Is a Known Risk Category

The OWASP Gen AI Security Project lists Excessive Agency as one of its top risks for LLM applications. It defines the risk as the vulnerability that allows damaging actions in response to unexpected, ambiguous, or manipulated model outputs, whatever the cause, and names three root causes: “excessive functionality; excessive permissions; excessive autonomy.”¹⁷ Its mitigations translate directly to quality systems:

• Minimize the tools and functions available to the agent. An agent that drafts investigation reports needs to read records and write drafts. It does not need a tool that can change record status.
• Minimize permissions. The agent’s service account in the QMS should have exactly the rights its oversight level allows, and no more.
• Require human approval for high-impact actions. OWASP’s guidance is to “require a human to approve high-impact actions before they are taken,” and notes this can be implemented in the downstream system.¹⁷
• Use complete mediation. OWASP recommends implementing authorization “in downstream systems rather than relying on an LLM to decide if an action is allowed or not.”¹⁷
• Log and monitor agent actions. Logging does not prevent excessive agency, but it limits damage by making unwanted actions visible.

Do Not Let the Agent Decide What Is Consequential

Feng and colleagues raise a design problem at their approver level that applies directly to quality: an agent may be instructed to seek approval before consequential actions, but reliably determining which actions are consequential may be challenging.³ In a quality system, the list of consequential actions should never be left to the agent’s judgment. It should be defined in your procedures, mapped to specific transactions in each system, and enforced by those systems. The four-question test earlier in this article is one way to build that list.

Give the Agent Its Own Identity

An agent should act under its own non-human account, never under a person’s credentials. That keeps the audit trail honest about who did what, and it keeps Part 11 signatures tied to the individuals who executed them.⁹ It also makes the permission model visible: an auditor can look at the agent’s account and see exactly what it is able to do.

Assign Roles for the Human Side

The NIST AI Risk Management Framework includes two items that fit here. GOVERN 3.2 calls for policies and procedures that define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems. MAP 3.5 calls for processes for human oversight to be defined, assessed, and documented.¹⁸ For a quality organization, that becomes a simple document: for each agent, which tasks it performs at which level, which role reviews or decides at each hand-off, and how that review is monitored.

A Practical Starting Sequence

For quality leaders who want to move from pilot to controlled use, we suggest the following sequence. It is deliberately conservative at the start, because the fastest way to lose the organization’s trust in agents is an inspection finding in the first year.

This sequence is slower at the start than a broad rollout, and that is deliberate. In regulated work, the controls designed in at the beginning are what keep a useful agent in production after its first inspection. An agent that is permitted to do less, and can show exactly what it did and who decided, is easier to defend, easier to extend, and easier to trust across the quality organization.

Questions to Ask a Vendor Before You Buy

Many quality teams will meet agents first as features inside an eQMS, LIMS, or document management product rather than as something they build. The same framework applies, and a few questions make the conversation concrete:

• Which actions can the agent take in the system, and can each one be switched off or limited by role?
• Does the agent act under its own account, or under the logged-in user’s account?
• Can the system enforce human approval on specific transactions, independent of the agent’s instructions?
• What model does the agent use, can its version change without notice, and how will we know?
• Does the audit trail show what the agent proposed separately from what the person decided?
• Can the agent return “undecided” or “insufficient evidence,” or does it always give an answer?

A vendor who can answer these clearly has thought about regulated use. A vendor who answers with accuracy figures alone has not yet been asked the right question.

Conclusion

Agentic AI in quality operations does not need a new theory of accountability. The accountability is already written down. The quality control unit approves and rejects. The QP certifies. An electronic signature belongs to one person. The draft Annex 22 keeps large language models out of critical GMP applications and puts a qualified person behind every output in non-critical ones. What quality leaders need is a consistent way to apply those fixed points to the many small steps inside each process. A four-level oversight model, sorted by a four-question test that starts with accountability rather than accuracy, gives them that. Agents can gather, draft, monitor, and recommend. People decide, approve, release, and sign. And the line between the two is enforced by system permissions, not by the agent’s good behavior.

Sakara Digital works with pharma and biotech organizations on AI strategy and governance for regulated operations, including where agents fit in quality systems and how to control them. If you are deciding where autonomy belongs in your deviation, CAPA, or release processes and want an independent view on where to start, we are happy to have that conversation.

For Further Reading