What an Agent Adds to Regulatory Intelligence

Regulatory intelligence (RI) is the work of finding new and changed regulatory requirements and expectations, working out what they mean for the company, and making sure someone acts. For most pharma and biotech companies it still runs on email subscriptions, a few vendor feeds, and a spreadsheet kept by a regulatory affairs or quality person who knows which sources matter. The tracking step has been automated for years. The reading, comparing, and assessing steps mostly have not.

Our earlier article on regulatory intelligence automation covered the vendor market and where automation helps or falls short in general. This article is narrower. It is about an agentic setup that does five jobs in sequence, and about the controls each job needs so that a quality unit can rely on the output.

1

Watch

Check a defined list of regulator channels on a schedule and record every item seen, including items judged not relevant.

2

Fetch and Store

Download the document itself, not just the announcement, and store a fixed copy with a content hash, retrieval time, and source URL.

3

Compare

Find the prior version the company last assessed, compute the differences, and describe what changed in plain terms.

4

Assess

Propose which SOPs, products, systems, and filings may be affected, with a cited reason for each one.

5

Route

Open a draft change record in the quality system, assign a human owner, and track it until that person accepts, rejects, or closes it.

Workflow or Agent: Decide Which Parts Get Freedom

The word “agentic” covers a wide range of designs. Anthropic’s engineering guidance draws a useful line between two kinds of agentic system: “Workflows are systems where LLMs and tools are orchestrated through predefined code paths,” while agents direct their own steps and tool use.1 The same guidance recommends starting with the simplest solution that works and adding complexity only when it is needed.

That distinction matters in a regulated setting. Watching, fetching, storing, and computing differences are well-defined tasks with right and wrong answers. They belong in fixed code paths that behave the same way every run and can be tested like any other software. The parts that benefit from a language model’s flexibility are narrower: deciding whether a new item is relevant to the company, describing a change in plain language, and matching a change to the company’s own documents. Giving the model freedom over which sources to check, or whether to skip a document it finds hard to read, adds risk without adding value.

A useful rule for design reviews: anything that decides whether an item enters the process should be deterministic and logged. Anything that decides what an item means can use the model, as long as a person checks the result and every statement points back to source text.

Why the Stakes Are Higher Than for an Alert Feed

ICH Q10, the pharmaceutical quality system guideline, lists the factors management can monitor, starting with “Emerging regulations, guidance and quality issues that can impact the Pharmaceutical Quality System”.2 An alert feed that misses a document leaves a gap that a person may still catch. An agent that has been trusted to watch, compare, and assess creates a different habit: people stop looking themselves. If the agent then misses a source or misdescribes a change, the error can pass into an impact assessment, a closed change record, and an SOP that is out of step with current expectations, all with a clean audit trail. The controls in the rest of this article are designed for that failure mode.

Start With a Source Register, Not a Model

The first design document for an RI agent should not be a prompt. It should be a source register: a controlled list of every channel the agent watches, what each channel publishes, what it does not publish, and how the agent will know when it has missed something. Each regulator publishes in its own way, and several of the gaps are written into the regulators’ own procedures.

FDA: Two Lists, and Guidance That Never Appears in the Federal Register

FDA’s Good Guidance Practices regulation, 21 CFR 10.115, defines two levels of guidance. Level 1 documents set out initial interpretations, changes in interpretation or policy of more than a minor nature, complex scientific issues, or highly controversial issues. Level 2 documents cover existing practices or minor changes in interpretation or policy.3 The procedures differ. For Level 1 documents, the regulation includes publishing a Federal Register notice announcing availability. For Level 2 documents, the listed steps are posting the document on the internet, implementing it immediately unless FDA says otherwise, and inviting comment. A Federal Register notice is not among those steps.

The practical consequence is simple. An agent that watches only the Federal Register for FDA guidance will miss Level 2 documents by design. The same regulation commits FDA to keep a current online list of all guidance documents and states that “New documents will be added to this list within 30 days of issuance.”3 That online list, which FDA presents as a searchable table filterable by product, date issued, organizational unit, document type, and draft or final status,4 is the better primary source for FDA guidance. CDER also keeps a sortable page of newly added guidance documents showing topic, title, draft or final status, and date.5 Note the 30-day allowance: the list is allowed to trail the issue date, so a reconciliation check against it should look back further than a day or a week.

CDER also publishes an annual guidance agenda. For 2026, CDER describes it as the list of guidances it is considering for development during the year, and states that it is not bound by the list and may develop guidance on topics not on it.6 The agenda is useful for planning which SOP owners should expect work. It is not a completeness control, because FDA says plainly that it is not bound by the list and may issue guidance on topics that are not on it.

The Federal Register: A Good API With Two Traps

The Federal Register is the right source for FDA rules, proposed rules, and notices, including notices of availability for Level 1 guidance. FederalRegister.gov offers a public REST API that returns JSON or CSV and needs no API key.7 The same developer page includes two warnings that belong in any source register. First, results are paginated, and “you can only paginate through the first 2000 results”; larger pulls must be split by date range. Second, the data comes from the files used to produce the printed edition, which can contain errors such as misspelled agency names that need cleanup. The page also notes that FederalRegister.gov is not the official legal edition, which is on GPO’s govinfo.gov.

Volume is manageable but not trivial. A query we ran against the API on October 29, 2026 returned 576 documents from FDA published in calendar year 2025, 481 of them notices.8 An agent that filters on the word “guidance” in the title, or on a single agency facet, will see a different number. The register should record the exact query the agent uses, so that a reviewer can rerun it and get the same population.

EMA: Publication Date Is Not Effective Date

EMA publishes RSS feeds for, among other things, public consultations, regulatory and procedural guidelines, scientific guidelines, and a general “What’s new” feed.9 An agent should subscribe to the specific guideline and consultation feeds rather than rely on news releases, which cover only a fraction of guideline activity.

The harder issue with EMA documents is status. EMA’s procedure for EU guidelines describes a sequence from concept paper through draft, public consultation, adoption, and implementation. Concept papers are normally open for consultation for two to three months and draft guidelines for three to six months. On timing, the procedure states: “Unless otherwise indicated, guidelines come into operation six months after their adoption.”10 The same procedure notes that the reference numbers of the concept paper, draft, final guideline, and revisions help track evolving documents. An agent that treats publication as the effective date will tell SOP owners to act too early, or will close the item before the date that matters. The register should require the agent to capture, for EMA items, the document reference number, the stage, the adoption date, and the stated date for coming into effect.

MHRA: Major and Minor Updates on GOV.UK

MHRA publishes on GOV.UK, which offers a Content API that returns JSON for any page on http://www.gov.uk. The API documentation labels it beta software. The reference documentation describes two update fields: public_updated_at, which is set automatically each time an edition is published as a major edition unless the publishing application sets it, and a separate updated_at field.11 A page can therefore change without its public update date moving.

The difference shows up in real records. When we pulled the GOV.UK content record for MHRA’s GxP data integrity guidance on October 29, 2026, it showed a first publication date of March 9, 2018, a public update date of September 27, 2021, two public change notes, and a separate internal update timestamp from October 2026.12 We are not suggesting the guidance text changed in October 2026. The point is that a monitor keyed on one date field sees a different history from a monitor keyed on the other, and the register must say which one the agent uses and why. For MHRA content, the safest pattern is to hash the attached documents themselves and treat a change in the hash as the trigger, with the change notes as supporting evidence.

PMDA: The Japanese Text Governs

PMDA publishes English translations of many ordinances and notices, and it is clear about their status. Its page of ministerial ordinances states that the translations are reference material and that “In the event of inconsistency between the Japanese originals and the translations, the former shall prevail.”13 Translations can also trail the original. An agent watching only PMDA’s English pages is watching a delayed, non-binding copy. If Japan is in scope for the company’s products, the register should list the Japanese-language source as primary, with the English page as a convenience, and the comparison step should run on the Japanese text.

ICH: One Guideline, Several Dates

ICH guidelines reach Step 4 when ICH regulatory members adopt them, and then each region implements them through its own procedures. The result is one guideline with several dates that matter to different teams. ICH M11, the Clinical Electronic Structured Harmonized Protocol (CeSHarP), shows the pattern.

EventDateSource
Adoption by ICH regulatory members under Step 4November 19, 2025EMA Step 5 technical specification, document history14
Final adoption by EMA’s CHMPDecember 11, 2025EMA Step 5 technical specification14
FDA guidance for industry (M11 Template)Dated May 2026FDA guidance document15
Date for coming into effect in the EUJune 11, 2026EMA Step 5 technical specification14

An agent that records only “ICH M11 finalized, November 2025” gives the clinical operations SOP owner the wrong date for every region they care about. The register should define ICH items as parent records with one child record per region, each with its own status and date.

Putting It in One Table

SourcePrimary channel for the agentKnown gap to design aroundField the agent must capture
FDA guidanceOnline guidance list and center pages of newly added guidanceLevel 2 guidance has no Federal Register notice; the list may trail issue by up to 30 daysLevel where stated, draft or final, issue date, center, docket number
Federal RegisterREST API with a fixed, stored query2,000-result pagination limit; source data errors; not the official legal editionDocument number, type, publication date, effective date, comment date
EMAScientific guideline and consultation RSS feedsPublication date differs from date for coming into effectReference number, stage, adoption date, effective date
MHRAGOV.UK Content API and attached filesMinor edits may not change the public update dateBoth update fields, change notes, attachment hashes
PMDAJapanese-language pages, English as convenienceEnglish translations are reference only and may lagJapanese title and date, English title if any, translation date
ICHICH guideline pages plus each region’s implementation pageStep 4 date differs from regional effective datesStep, Step 4 date, regional status and date per region
30 daysTime FDA allows itself to add a new guidance document to its online list after issuance3
2,000Maximum results the Federal Register API lets a client page through for one query7
6 monthsDefault time after adoption before an EU guideline comes into operation10

Completeness Checks: How You Know Nothing Was Missed

No monitoring design can promise that it will never miss a document. What it can do is make misses detectable, quickly, by someone other than the component that missed them. That is the honest meaning of “without missing one”: not a claim of perfection, but a set of checks that would expose a miss within a known window.

The core idea. Never let the agent be the only witness to its own coverage. Every completeness check compares what the agent saw against a list the agent did not produce.

Six Checks That Work in Practice

1

Channel Heartbeat

For every channel in the register, log each run with a timestamp, HTTP status, and item count. A channel that returns zero items for longer than its normal quiet period, or that starts returning errors, raises an alert to a person. A broken feed often looks exactly like a quiet week.

2

Count Reconciliation

Each week, rerun the stored query for each source over a trailing window and compare the count and identifiers with what the agent ingested. For FDA guidance, use a window of at least 45 days so the 30-day listing allowance is covered. Any identifier in the source that is missing from the agent’s log is an exception for review.

3

Cross-Channel Matching

Match items across channels that should agree. A Federal Register notice of availability for an FDA guidance should have a matching entry on the FDA guidance list. An ICH Step 4 adoption should, in time, produce regional implementation records. Unmatched items point to a gap in one channel or the other.

4

Hash Sweep of Assessed Documents

Periodically refetch every document the company has already assessed and compare its hash with the stored copy. This catches revisions that were posted in place without an announcement, which is the hardest kind of change to see.

5

Independent Sample

Once a month, a person independently reviews one source for the period, using the regulator’s own website, and compares the result with the agent’s log. Rotate the source each month. This is the only check that tests the register itself, because it can reveal a channel that should have been in the register and was not.

6

Seeded Test Items

Keep a set of known past revisions and run them through the pipeline after any change to the agent, its prompts, or its model. If a known revision no longer produces the expected detection and comparison, the change does not go live.

Record the Negatives, Not Just the Hits

An RI log that lists only relevant items cannot prove coverage. If an inspector or auditor asks whether the company saw a particular guidance and decided it did not apply, the answer needs a record: the item was ingested on a given date, the relevance screen marked it out of scope for a stated reason, and a person confirmed that decision or the decision fell under a pre-approved rule. A “not relevant” decision is a decision, and in a regulated company decisions need records.

This has a practical design consequence. The relevance screen should never discard items without a record. It should sort them into “relevant”, “not relevant with reason”, and “unsure”. Items marked unsure go to a person. Items marked not relevant are sampled on a schedule so that the screen’s error rate is known rather than assumed.

Measures Worth Reporting

Four numbers tell a quality or regulatory leader whether the monitoring is working: the share of reconciliation exceptions closed within the target period; the time from regulator publication to agent ingestion, by source; the share of sampled “not relevant” decisions that a reviewer overturned; and the number of channels with a failed heartbeat in the period. None of these measure the model’s intelligence. All of them measure whether the company can show it is not missing things.

A common mistake. Teams often test an RI agent by checking whether its summaries of documents it found are good. That tests the comparison step and says nothing about coverage. A coverage test starts from the regulator’s list and asks whether each item reached the agent, not the other way around.

Comparing Versions Without Inventing Changes

The comparison step answers the question the business cares about most: what is different from the version we last assessed? It is also the step where language models make a specific, well-documented kind of error.

What the Research Shows About Edit Detection

A 2024 study from the Technical University of Munich and Adobe Research tested GPT-4 and GPT-3.5 on summarizing the changes between two versions of a document. The authors found that the models failed to correctly spot the edits, a problem they called edit hallucination: the model presented a span of text as an edit when it was not an actual modification, and sometimes produced text that was not in the original document. Their fix was to give the model input in a diff format, with each edit explicitly marked and numbered, so that the model describes known edits rather than finding them.16 They also found that the problem grew worse with longer documents when chunking misaligned the two versions.

A separate line of research explains why long documents are risky. The “Lost in the Middle” study found that language model performance can drop significantly depending on where relevant information appears in a long input, with performance often highest when the information is at the beginning or end and worse when it is in the middle.17 Regulatory guidance documents are long, and the substantive revision in a 40-page update is often in the middle.

The Design Rule: Software Finds the Differences, the Model Describes Them

Taken together, the evidence points to a clear split of work.

DETERMINISTIC

Text Extraction and Alignment

Extract text from both versions with the same tool and settings. Align by section number and heading where the document has them, not by page. Keep the extraction output as a stored record.

DETERMINISTIC

Difference Computation

Compute insertions, deletions, and modifications with an ordinary diff tool. Give each change an identifier, its location in both versions, and the exact old and new text.

MODEL-ASSISTED

Change Description

Ask the model to describe each numbered change in plain language and to classify it, for example editorial, clarification, new expectation, removed expectation, or changed timeline. The model may not add changes that are not in the computed list.

MODEL-ASSISTED, HUMAN-CHECKED

Grouping and Significance

Let the model group related changes into themes and suggest which are substantive. A reviewer confirms the grouping and the significance before anything moves to impact assessment.

With this split, the comparison has a property a reviewer can test: every change the model describes must map to an identifier in the computed diff, and every identifier in the computed diff must have a description or an explicit “editorial only” label. A simple script can check both directions. If the model’s output mentions a change with no matching identifier, the output fails and goes to a person.

Problems That Are Not About the Model

Many comparison errors come from earlier in the chain. PDF text extraction can split words, merge columns, drop tables, or reorder footnotes, and a different extraction tool on the second version can produce hundreds of false differences. Regulators sometimes restructure a guidance, renumber its sections, or move content to a question-and-answer annex, which makes a section-by-section comparison misleading. And some of the most important changes are not in the body text at all. A move from draft to final, a new effective date, or a withdrawal notice may appear only in metadata, on a cover page, or on the web page that links to the file.

The comparison record should therefore include three parts: the body text differences, the metadata differences (status, dates, reference numbers, title), and a note of any structural change that makes a line-by-line comparison unreliable. When the structure has changed, the honest output is “restructured; full reread required”, routed to a person, rather than a confident summary built on a misaligned diff.

Choosing the Right Baseline

“Compare with the prior version” sounds simple, but the prior version that matters is the one the company last assessed, which may not be the regulator’s immediately previous version. If a guidance was revised twice since the company’s last assessment, comparing only the last two regulator versions hides the first revision. The agent should store, for every assessed document, the exact copy that was assessed and the date of that assessment, and compare new versions against that stored copy. This is also why stored copies matter: regulators do not always keep earlier versions online.

Citations Back to the Source Text

Every statement the agent makes about a regulatory document should point to the exact passage that supports it. This is the control that turns an agent’s output from something a reviewer must take on trust into something a reviewer can check in minutes.

Why Retrieval Alone Does Not Solve It

Grounding a model in retrieved documents reduces invented content, but it does not remove it. A Stanford study of AI legal research tools built on retrieval-augmented generation, published as a preprint in 2024, found that the tools tested from LexisNexis and Thomson Reuters each hallucinated between 17% and 33% of the time, despite vendor claims about avoiding hallucinations. The study counted as hallucinated any response containing a false statement or falsely stating that a source supports a statement.18 Legal research is a close match for regulatory reading: long, precise texts where the difference between “should” and “must” matters.

Citation quality is its own problem. The ALCE benchmark from Princeton, which measures whether model-generated citations support the statements they are attached to, found that even the best models tested lacked complete citation support 50% of the time on one of its datasets.19 A citation that points to the right document but the wrong passage, or to a passage that does not say what the statement claims, gives a reviewer false confidence.

17% to 33%Hallucination rate per tool for the RAG-based legal research tools tested from two major providers18
50%Share of the time the best models tested lacked complete citation support on ALCE’s ELI5 dataset19
576FDA documents in the 2025 Federal Register returned by our API query on October 29, 20268

What a Usable Citation Contains

A citation in an RI agent’s output should let a reviewer go straight to the evidence without searching. In practice that means five fields:

  • Document identifier. The regulator’s own reference: docket number, EMA reference number, Federal Register document number, or GOV.UK content identifier.
  • Version identifier. The content hash and retrieval date of the stored copy the statement was drawn from.
  • Location. Section number and heading, plus page number for PDFs, in the stored copy.
  • Quoted span. The exact sentence or clause from the source, copied character for character.
  • Diff identifier. For statements about what changed, the identifier of the computed change the statement describes.

Check Citations Mechanically Before a Person Sees Them

Because the quoted span is exact text, software can verify it. Before an output reaches a reviewer, a check confirms that each quoted span appears verbatim in the stored copy at the stated location. If it does not, the statement is flagged. This check is cheap, it catches the most damaging errors, and it does not depend on another model’s judgment.

The mechanical check proves the quotation is real. It does not prove the quotation supports the statement. That second judgment belongs to the reviewer, and the output format should make it easy: the statement on the left, the quoted source text on the right, the link to the stored copy underneath. A reviewer who can see both side by side can confirm or reject a statement in seconds. A reviewer who must open a 60-page PDF and search for the relevant passage will, after the tenth item of the week, start trusting the summary instead.

Rules for quotations in RI output.

  • A quotation must match the stored source exactly, including “should”, “must”, “may”, and “can”.
  • A paraphrase is never shown inside quotation marks.
  • If the source does not state something, such as a date or a number, the output says so rather than estimating it.
  • Statements about effective dates cite the regulator’s own statement of the date, not a news article about it.

Keep the Source Copies

Citations point to stored copies, so the stored copies are records. They should be kept for as long as the decisions that depend on them, with access controls and a retention rule that matches the company’s records policy. Pointing citations only to live URLs is a weak design: regulators reorganize websites, revise documents in place, and retire old versions, and a citation to a page that has changed no longer proves what the reviewer saw.

From Change to Impact: SOPs, Products, and Filings

Knowing that a guidance changed is only useful if the company knows what the change touches. Impact assessment is where an agent can save the most effort and where its suggestions need the most careful framing.

Impact Assessment Depends on Your Own Data

An agent can only match a regulatory change to the company’s documents if the company’s documents are described well enough to match against. That usually means three inventories: the controlled SOP index with each document’s scope, owner, and the regulations and guidance it relies on; a product register with dosage forms, markets, and lifecycle stage; and a system inventory for computerized systems in scope of GxP. Where these inventories are incomplete, the agent’s impact suggestions will be incomplete in the same places. Many companies find that building the agent exposes gaps in these inventories, which is worth fixing for its own sake.

The most useful single field to add, if it is missing, is a list of the external requirements each SOP implements. When an SOP records that it implements, for example, a specific section of a specific guidance, the agent can match changes to that section directly. When it does not, the agent falls back on matching by topic and wording, which produces more false matches and more misses.

Ask for High Recall, Then Let People Narrow

For impact assessment, a missed impact is worse than an extra suggestion. The agent should be tuned to suggest every SOP, product, and system that may be affected, with a cited reason for each, and a reviewer should narrow the list. An agent tuned for a short, tidy list will look better in a demonstration and fail in practice, because the items it leaves off never reach a person.

Each suggested impact should carry:

  • the computed change identifier and its quoted text;
  • the affected SOP, product, or system, with the specific section where relevant;
  • the reason for the match in one or two sentences, with the quoted text from the company document;
  • a suggested category, such as no action, training only, procedure revision, system configuration change, or possible regulatory filing; and
  • the person the agent proposes as owner, taken from the SOP index or product register rather than inferred.

The Filing Question Belongs to Regulatory Affairs

ICH Q10 says proposed changes should be evaluated against the marketing authorization, and that there should be an assessment of whether a change to the regulatory filing is required under regional requirements.2 An agent can flag that a change may touch a filed commitment, a specification, or labeling, and it can point to the relevant section of the filing if the filing content is in a system it can read. It should not decide the filing category. That judgment depends on regional variation rules, the history of the product, and conversations with the agency that are rarely all written down in one place. The agent’s job is to make sure the question reaches a regulatory affairs owner with the evidence attached.

“No Impact” Is a Conclusion That Needs an Owner

Most regulatory changes will not affect most SOPs. That is exactly why “no impact” conclusions are risky: they are frequent, they look routine, and they are the easiest to approve without reading. A sound design requires a named person to confirm each “no impact” conclusion for any item the relevance screen marked relevant, with the reason recorded. For items below a defined risk threshold, the company can pre-approve a rule, for example that a Level 2 FDA guidance on a product class the company does not make is closed as not applicable, but the rule itself should be approved in advance and the closures sampled.

Routing Into Change Control and Who Signs

The last step is where the agent’s work meets the quality system. It is also where the design question that matters most is the simplest one to state: who signs?

What the Regulations Already Say

For drug manufacturing in the United States, 21 CFR 211.100(a) requires written procedures for production and process control and states: “These written procedures, including any changes, shall be drafted, reviewed, and approved by the appropriate organizational units and reviewed and approved by the quality control unit.”20 ICH Q10 adds that proposed changes should be evaluated by expert teams with the relevant expertise, for example from pharmaceutical development, manufacturing, quality, regulatory affairs, and medical, so that the change is technically justified.2

Neither text leaves room for a tool to approve. An agent can draft. It can propose. It can assemble the evidence. The review and approval of the change, and of the impact assessment that justifies it, belong to the appropriate organizational units and the quality unit, which means named people.

What a Signature Means

Where the change record is electronic, 21 CFR 11.50 requires that signed records show the printed name of the signer, the date and time of signing, and the meaning of the signature, such as review, approval, responsibility, or authorship.21 That last element is useful for agent design. The agent’s contribution can be recorded honestly as drafted content, with the agent’s version and the run identifier as metadata. The human signature then carries a clear meaning: “I reviewed this assessment against the cited sources and I approve it.” The record should never present agent-drafted text as if a person authored it, and it should never present a person’s approval as if it were a review of something they did not see.

A Sign-Off Matrix

StepAgentRI analystProcess or SOP ownerRegulatory affairsQuality assurance
Ingestion and relevance screenPerforms and logsReviews “unsure” items and samples “not relevant”NoneNoneApproves screening rules
Version comparisonComputes diff and drafts descriptionsVerifies descriptions against diff and sourceNoneConsulted on status and datesNone
Impact assessmentDrafts candidate impacts with citationsChecks completeness of candidatesSigns the assessment for their processSigns the filing assessmentReviews
Change recordOpens draft and proposes ownerConfirms routingOwns and implementsOwns filing actionsApproves and closes
“No impact” closureProposes with reasonConfirms below-threshold items under approved ruleSigns where item was marked relevantConsulted where filing is touchedSamples closures

The Person Who Signs the Impact Assessment

The impact assessment should be signed by the person accountable for the affected process, not by the RI analyst who ran the intake and not by a central reviewer who signs everything. The RI analyst’s job is to confirm that the agent’s output is accurate and complete: the right document, the right version, the right changes, real citations. The process owner’s job is to decide what the change means for work they are responsible for. Combining the two in one signature makes it easy for either check to be skipped.

Two practical points follow. First, the agent should take the owner from the SOP index or product register and never guess. If the record has no owner, that is an exception for a person to resolve. Second, routing should respect the change control system’s existing categories and timelines rather than creating a parallel queue. A regulatory-driven change is still a change, and it should compete for priority on its merits in the same system as every other change. Our article on clearing a change control backlog covers what happens when parallel queues build up.

A good test of the design. Pick any closed regulatory-driven change record and ask: Can a reviewer see which regulator document and which version triggered it, which computed changes were judged relevant, the exact source text for each, which SOPs were considered and rejected and why, and who signed each decision and with what meaning? If all of that is visible from the record in a few minutes, the design is working.

Validating and Running the Agent Itself

An RI agent that feeds change control is part of the quality system’s inputs. How much validation it needs depends on its intended use and on the controls around it, and companies should decide that in writing before the agent goes live.

Where It Falls in FDA’s AI Thinking

FDA’s January 2025 draft guidance on using AI to support regulatory decision-making for drugs and biologics, which remains a draft, describes a risk-based credibility assessment for AI models that produce information or data to support regulatory decisions about safety, effectiveness, or quality. Its stated scope excludes AI used for operational efficiencies, such as internal workflows, resource allocation, and drafting a regulatory submission, that do not affect patient safety, drug quality, or the reliability of study results.22 Many RI agents will fall on the operational side of that line, because a person makes and signs every decision. That does not remove the need for control. The agent still produces GxP-relevant content that people rely on, and the company’s computerized system procedures still apply. The draft guidance’s encouragement to engage FDA early where scope is uncertain is a reasonable posture for unusual designs.

What to Test Before Go-Live

Risk-based validation for an RI agent should focus on the failure modes described above rather than on generic model quality. A practical test set includes:

  • Coverage tests. For each source in the register, a past period where the full list of items is known. The agent must ingest every item or log an exception for it.
  • Comparison tests. Pairs of real past versions with known changes, including at least one restructured document, one change only in metadata such as draft to final, and one long document where the main change is in the middle.
  • Citation tests. Outputs checked for verbatim quotations, correct locations, and statements that the quoted text supports.
  • Impact tests. Past regulatory changes where the company’s actual assessment is on record, compared with the agent’s candidate list. The measure that matters is how many of the real impacts the agent included.
  • Routing tests. Change records opened in a test instance of the quality system with the right owner, category, and links, and failure cases such as a missing owner handled as exceptions.

Running It: Monitoring and Change Control for the Agent

Once live, the agent itself needs change control. A new model version, a revised prompt, a new extraction library, or a change to a source’s API can each change behavior. The seeded test items from the completeness section serve as a regression suite: run them after every change, and do not release a change that breaks them. The reconciliation exceptions, overturned “not relevant” decisions, and failed citation checks form the ongoing performance record. Review them at a set interval, and treat a trend in any of them as a signal to investigate.

Finally, keep the people skilled. An agent that does the reading can erode the team’s own familiarity with regulator websites and document styles. The monthly independent sample is a control, and it is also a way to keep RI analysts familiar with the sources, so that when the agent is wrong, someone on the team notices.

Conclusion

Agentic AI regulatory intelligence is worth building because the reading, comparing, and matching work it takes on is slow, repetitive, and easy to get wrong when people are busy. But the value depends on controls that have little to do with the model. The source register has to reflect how each regulator publishes, including the gaps written into their own procedures. Completeness has to be checked against lists the agent did not produce. Differences have to be computed by software and only described by the model. Every statement has to point to an exact, stored passage. And every decision has to be signed by the person accountable for the process it affects, with the agent’s contribution recorded for what it is.

Sakara Digital works with pharma and biotech organizations designing regulatory intelligence and change control processes that hold up under inspection, including where AI agents do part of the work. If you are planning an agentic RI setup, or trying to decide how much of one your quality system can rely on, we are happy to talk it through and offer an independent view on where to start.

For Further Reading