From RPA to Agentic: Retiring a Bot Estate Without Breaking Validation

How the Bot Estate Turned Into a Liability

The original case for robotic process automation was straightforward and, for a while, correct. A bot could sit on top of systems that would not talk to each other, imitate the keystrokes a person used to make, and run overnight without complaint. No integration project, no vendor negotiation, no change to the underlying application. Finance signed the business case in a single meeting. Quality signed off on a narrow scope. The first ten bots paid for themselves.

Adoption was broad. Deloitte’s intelligent automation survey found 74 percent of responding organizations already implementing robotic process automation, with half also implementing optical character recognition and a further group planning process mining work.¹ Pharma joined that wave with its own pattern: bots reconciling batch records against enterprise resource planning entries, bots pulling adverse event data from partner portals, bots renaming and filing study documents, bots keying the same lot number into three systems because nobody had funded the interface.

What did not scale was the estate. The same Deloitte work reported 62 percent of organizations naming integration difficulty as a barrier to scaling, 55 percent citing missing skills, and 52 percent struggling with process and workflow change. Expected payback stretched from 16 months in the earlier survey round to 22 months in the later one.¹ An industry survey cited in ISPE’s Pharmaceutical Engineering put the share of organizations with scaled-up, industrialized robotic process automation at only 13 percent.² Most estates were never industrialized. They accumulated.

The specific failure mode of a bot estate

Bots fail differently from applications. An application breaks when its own code is wrong. A bot breaks when something it does not own changes underneath it. A vendor pushes a user interface update and the button moves twelve pixels. A field label changes from “Batch” to “Batch ID”. A browser update alters how a page renders. None of these are defects in the bot. All of them stop it.

Academic work on robotic process automation has documented this pattern. A systematic review of the field identified the absence of a disciplined method for designing, executing, and operating bots across their life cycle as one of the central open problems, noting that practical adoption ran well ahead of the research base.³ Later work applying statistical process control to bot execution logs found that variability in the surrounding environment strongly influences bot behavior and performance, and proposed monitoring bot runs the way you would monitor a manufacturing process, so that erratic behavior surfaces before it becomes a failure.⁴ Research applying bots to monitor hospital information systems reached a related conclusion: bots reveal the end user experience that conventional infrastructure monitoring misses, precisely because they interact with the surface the user sees.⁵

Translate that into an operating burden. Every upstream release becomes a potential bot outage. Every bot outage in a GxP process becomes a deviation, an investigation, and possibly a manual workaround executed under time pressure. The automation team stops building and starts patching. In a regulated setting the patching is not just an engineering fix, because each change to a validated bot passes through change control, testing, and approval.

Why agentic AI arrived as the answer

Vendors read the same room. If the problem with a bot is that it is rigid, the pitch writes itself: an agent reads meaning rather than screen positions, so it adapts when the screen changes. That claim is real in some settings, which is what makes the pitch effective. It is also being applied far beyond those settings. Gartner has forecast that more than 40 percent of agentic AI projects will be canceled by the end of 2027, driven by escalating spend, unclear value, and inadequate risk controls, and has warned about “agent washing”, the rebranding of existing assistants, chatbots, and robotic process automation products as agentic without substantive change.⁶

For a regulated organization, agent washing is more than a procurement annoyance. If a vendor relabels the automation you already own, you may find yourself revalidating a product that has not functionally changed, or worse, accepting a genuinely non-deterministic component into a process whose validation package assumes a deterministic one. The first wastes effort. The second creates a compliance gap that will not be visible until an inspector asks how you established the system performs as intended.

Start With Triage, Not Migration

The instinct on receiving a mandate to modernize automation is to inventory the bots and start estimating replacement effort. That sequence guarantees the wrong answer, because it assumes every bot deserves a successor. Start instead by asking a blunter question of each bot: should this work happen at all?

In practice, running that question across an estate produces four groups, and only one of them is a migration.

The evidence that decides each case

Sorting cannot be done from a spreadsheet of bot names. Each bucket has its own evidence test, and each test is cheap relative to the effort of building a replacement.

For retirement, pull the bot’s execution logs and the downstream consumption of its output. A bot that writes a file nobody opens, or that populates a field no report reads, is producing motion rather than value. Ask the process owner to name the decision that depends on the output. If nobody can name one within a week, that is your answer. Retirement candidates also appear where a bot was built to reconcile two systems and one of those systems has since been decommissioned, leaving the bot faithfully reconciling against a stub.

For leave as is, look at change history and failure rate. A bot that has run without modification for eighteen months against a system on a stable release cadence is telling you something. It is deterministic, its validation package is current, and its operating burden is small. Replacing it introduces risk with no matching benefit. The discipline here is to write the decision down, with the evidence, so that a future modernization program does not rediscover the same bot and reopen the same argument.

For native functionality, go back to the platform vendor’s release notes for the period since the bot was built. Most enterprise platforms in this space have added integration surfaces, scheduled reporting, and workflow capability over the last five years. A bot that screen-scrapes a report which the vendor now exposes through a documented interface is not a modernization candidate. It is a configuration change. The validation effort is a fraction of building a replacement, and the result is more stable than either the old bot or a new agent.

For agent replacement, you need evidence of judgment. Look at the exception rate, at the volume of manual handling the bot generates, and at whether the inputs arrive as free text, scanned documents, email bodies, or partner formats that vary. If the bot completes 96 percent of transactions untouched and hands 4 percent to a person, the bot is doing well and the agent opportunity is in the exception queue, not the main path.

Governing the sort

The sort needs a decision forum with authority to retire work, not just to approve technology. That means a process owner who can say a step is no longer required, a quality representative who can confirm the step is not a documented control, and someone from IT who knows what the platform can now do natively. Without the process owner, everything defaults to replacement, because nobody in a technology meeting is comfortable deleting a control they do not own.

Record each decision with its evidence and its date. When an inspector asks why a validated GxP bot was switched off, “the process owner determined the reconciliation was superseded by the interface implemented in the 2024 upgrade, and quality confirmed it was not a documented control” is a complete answer. “It was part of the automation modernization program” is not.

The Determinism Problem at the Center of the Decision

Everything in this migration turns on one property, and most program plans do not name it.

A robotic process automation bot is deterministic. Given the same inputs and the same environment, it executes the same steps in the same order and produces the same output. That property is not incidental to how the bot was validated. It is the foundation of it. Your test scripts assert expected results for defined inputs. Your operational qualification demonstrates the sequence executes as specified. Your ongoing assurance rests on the assumption that nothing inside the bot decides anything. As the ISPE article on robotic process automation in pharma put it, these systems are configurable, auditable, and specifically designed not to color outside the lines.²

An agent that plans its own route to a goal does not have that property. Given the same inputs, it may take a different sequence of steps, call different tools, or produce a differently worded output that is equally correct. Even where the final answer is stable, the path is not. That is the entire point of using one. It is also the reason your existing validation approach does not transfer.

What the regulators have signaled

Regulators have been unusually direct about determinism. The draft Annex 22 on artificial intelligence, issued for consultation by the European Commission alongside PIC/S in July 2025, restricted its scope to static, deterministic models in critical GMP applications. Its glossary defines a static model as one where all parameters have been finally set and which does not adapt its performance during use by incorporating new data. Dynamic models, probabilistic outputs, generative models, and large language models were placed outside the scope for critical applications.⁷⁸ Commentary on the draft made the reasoning plain: a model that keeps learning in production cannot offer a stable basis for release decisions, because parameter drift can follow from abnormal or unrepresentative data.⁹

Following the consultation, the drafting group signaled that it was considering whether and under what risk-based safeguards adaptive and probabilistic models might be permitted, so the final position may soften. Plan on the current text. A program that assumes future relaxation is betting a validation strategy on a regulatory outcome it does not control.

The United States position is framed differently but points the same way. The FDA’s final guidance on computer software assurance for production and quality management system software, issued in September 2025, sets out a risk-based approach in which assurance effort scales with the risk the software poses to product quality and patient safety, and asks organizations to think about the intended use of each feature and function rather than testing everything to the same depth.¹⁰¹¹ That framework accommodates an agent, but only if you can articulate the intended use precisely and defend the assurance activities you chose for it. Vagueness about what the agent decides is not compatible with it.

Underneath both sits the predicate rule. 21 CFR 211.68 requires that automatic, mechanical, and electronic equipment used in manufacturing be routinely calibrated, inspected, and checked according to a written program designed to assure proper performance, with those activities documented.¹² A written program designed to assure proper performance is precisely what a non-deterministic component makes harder to write, and precisely what you must produce anyway.

Three practical consequences

First, your acceptance criteria change shape. A deterministic bot is tested against exact expected results. An agent is assessed against a defined range of acceptable outcomes, with criteria that specify what counts as correct, what counts as an acceptable variation, and what counts as a failure. Writing those criteria is the hardest part of the whole program, and it belongs to the process owner and quality, not to the vendor.

Second, your monitoring obligation grows. A bot that passes qualification will keep behaving the same way until something around it changes. An agent’s behavior can shift with a model update, a prompt change, a tool change, or a change in the distribution of inputs. That means ongoing performance monitoring is not an optional maturity item, it is part of the assurance basis. NIST’s generative AI profile for its AI Risk Management Framework treats pre-deployment testing, continuous monitoring, and incident disclosure as core measures precisely because behavior is not fixed at release.¹³

Third, human oversight has to be designed, not assumed. If the agent’s output feeds a GxP decision, someone has to be positioned to catch a wrong answer, with the information and the authority to do so. An oversight step that consists of a person approving whatever the agent produced is not oversight. Analysis of GxP and AI tooling has consistently made the same point: trust in these systems comes from the controls around them and the ability to demonstrate they were applied, not from confidence in the model.¹⁴

Where an Agent Genuinely Beats a Bot, and Where It Does Not

Determinism is a property, not a verdict. There are processes where losing it is a fair trade and processes where it is a straightforward downgrade. Being specific about which is which prevents most of the waste in these programs.

Three conditions where the agent wins

Exception handling. The classic bot design handles the happy path and routes everything else to a person. In many estates the exception queue is where the actual work sits: a supplier certificate arrives in a format the bot does not recognize, a batch record has a handwritten annotation, a partner sends a safety report with fields transposed. Exceptions are, by definition, the cases that were not enumerated in advance, which is exactly what a rule-based bot cannot handle and what an agent can attempt. The economics also favor it: automating the main path was already done, so the remaining benefit lives entirely in the exceptions.

Unstructured input. Where the input is free text, a scanned document, an email thread, or a document whose layout varies by sender, the bot needs a rigid template and breaks whenever the template is violated. An agent working from meaning rather than position handles variation that would otherwise require a new rule for every sender. Regulatory correspondence, investigator site communications, supplier documentation, and literature screening all sit in this category.

Brittleness that comes from the interface, not the logic. This is the most underrated case. Some bots are unstable not because the process is complicated but because the bot is reading a user interface that changes. If the logic is simple and the failures all trace to screen changes, the honest first question is whether an application programming interface now exists, which puts the bot in bucket three. If no interface exists and none is coming, an agent that can work from what the screen means rather than where the elements sit is a genuine improvement in stability.

Three conditions where the bot wins

High volume with fully structured input. If a bot processes forty thousand identical transactions a month from a fixed-format file, the agent brings nothing. The bot is faster, its per-transaction spend is lower, its behavior is reproducible, and its validation package is straightforward. Introducing variability into a process that has none is a net loss.

Stability. Where the surrounding systems change rarely and the bot has run untouched for a long period, the maintenance argument disappears. The whole case for replacement rested on maintenance burden. If there is no burden, there is no case.

Direct GxP criticality with a simple rule. Where a step directly affects product quality, data integrity, or patient safety, and the step can be expressed as an unambiguous rule, determinism is a feature you should not trade away. This is the space Annex 22 was drafted around.⁷ A rule you can state in one sentence should be implemented as a rule.

Reverse-Engineering the Requirement From the Bot

Here is the problem that derails schedules. A meaningful share of bots encode business logic that exists nowhere else. The specification, if one was written, described what the bot should do at a level far above what it actually does. The conditional branch that handles the German site differently, the hard-coded tolerance, the exclusion for one product family, the retry logic that quietly compensates for a slow interface: none of that is in a document. It is in the bot. And the person who built it left in 2022.

This is not a documentation hygiene complaint. It is a validation problem with a direct consequence. You cannot validate a replacement against a requirement you do not have, and you cannot demonstrate equivalence to behavior you have not characterized. So the migration does not begin with design. It begins with recovering the requirement from the artifact.

A method that works

Handle the undocumented control carefully

One category deserves particular care. Sometimes a rule recovered from a bot turns out to be a real GxP control that was never documented as one. A tolerance check, a data verification, a segregation of duties enforcement. The bot has been performing a control for years and your quality system does not know it exists.

The temptation is to quietly fold the rule into the replacement and move on. Resist it. That control needs to be documented, assessed, and brought into the quality system on its own terms, regardless of what happens to the bot. If it is a control, it needs an owner, a procedure, and an assurance approach. Discovering it during a migration is fortunate. Carrying it forward undocumented into an agent, where it becomes a non-deterministic control nobody has written down, is materially worse than where you started.

Building a Validation Basis for the Replacement

Once the sort is done and the requirements are recovered, the work divides cleanly by bucket, and so does the validation effort.

Retirements need a documented decision, a change control, an assessment of whether any GxP control is affected, and a records plan. They do not need validation, because nothing is being built. They do need the same rigor of justification as a build, which teams often forget.

Native functionality replacements are configuration changes to an already-validated platform. They go through that platform’s existing change control and are tested against the recovered requirement. Risk-based assurance applies in the ordinary way, with effort scaled to what the configuration affects.¹⁰ This is the cheapest and most durable outcome in the whole program, and it should be pursued aggressively.

Agent replacements need a validation approach built for a system that does not repeat itself exactly. That is the piece worth spelling out.

Five elements of an agent validation package

A precise statement of intended use. Not “handles supplier documentation” but a specific description of what the agent receives, what it decides, what it must never decide, and what it hands to a person. The narrower the statement, the smaller the assurance effort and the clearer the boundary. This is also where risk classification happens, because the risk is a property of what the agent decides, not of the technology.

Acceptance criteria expressed as ranges and rules. For each output type, define what correct means, what acceptable variation looks like, and what constitutes failure. Some outputs will be exactly checkable, for example an extracted lot number. Others will require a judgment rubric applied by a qualified reviewer. Both are legitimate. What is not legitimate is leaving the criteria unstated and assessing outputs by impression.

A test set that reflects real input distribution. Build it from the execution history you mined during requirement recovery, so it contains the real proportion of clean cases, edge cases, and the specific ugly cases that broke the bot. Hold part of it back and never use it during tuning, so you retain an honest measure. A test set assembled from convenient examples will pass and tell you nothing.

A defined human oversight point with real authority. Specify who reviews what, on what basis, and what happens when they disagree with the agent. Make sure the reviewer sees the information needed to disagree, including the agent’s reasoning and the source material. Record the disagreements. They are your best early signal of drift.

An ongoing monitoring plan tied to change control. Define what you measure in production, how often, and what threshold triggers investigation. Then connect it to change control so that a model version change, a prompt change, or a tool change is a controlled change with an impact assessment, not a silent vendor update. NIST’s guidance for generative systems treats pre-deployment testing plus continuous monitoring as a pair for exactly this reason.¹³

Decommissioning Discipline: Parallel Running, Equivalence, and Closure

Building the replacement is the visible half of the program. Retiring the bot properly is the half that gets audited. Four things need to be right.

Parallel running, designed rather than assumed

Run the old bot and the new implementation side by side, on the same inputs, for a defined period, with only one of them authoritative. In most cases the bot stays authoritative and the agent runs in shadow, because if the agent is wrong you want that to be a finding rather than a deviation.

Define the period by evidence, not by calendar convenience. The right length is whatever it takes to see the real range of inputs, including the month-end pattern, the quarterly pattern, and at least a few of the exception types that matter. A process with strong monthly seasonality needs to see a full cycle. Fix the criteria for ending parallel running before it starts, and write down what happens if the criteria are not met, because a parallel run with no defined exit becomes a permanent second system that nobody owns.

Proving equivalence when the new system is not deterministic

This is the technical heart of the decommissioning. You cannot demonstrate equivalence by comparing outputs character for character, because the agent will not produce character-identical output and does not need to. Comparing on that basis produces a mountain of false differences and destroys confidence in the exercise.

Compare at the level of the decision instead. Four tiers work well:

• Decision equivalence. Did both reach the same outcome? Same disposition, same routing, same accept or reject. This is the tier that matters most and the one to report to quality.
• Field-level equivalence for structured extraction. Where the output is a discrete value, for example a lot number, a date, or a quantity, exact match is appropriate and should be near total. Investigate every mismatch individually.
• Bounded equivalence for computed values. Where a number is derived, define the tolerance in advance based on what the downstream use requires, and treat anything outside it as a failure rather than a rounding curiosity.
• Rubric-scored equivalence for narrative output. Where the output is text, score it against a rubric covering completeness, accuracy, and absence of unsupported content. Use more than one reviewer on a sample and check that they agree with each other, because a rubric that reviewers apply inconsistently is not evidence.

Then investigate the differences rather than counting them. A 2 percent divergence rate is meaningless on its own. Two percent that is entirely one input type, where the agent is right and the bot was quietly wrong for three years, is a very different finding from 2 percent scattered randomly across the population. The first is a discovery about your old process. The second is an unexplained instability and a reason not to proceed.

Records from the retired bot

A bot in a GxP process generated records: execution logs, audit trails, outputs, and evidence that steps occurred. Switching the bot off does not switch off the retention obligation attached to those records, and the retention period is set by the predicate rule that applies to the process, not by the life of the software.

MHRA’s data integrity guidance is explicit about what archiving means in practice. Archived data must be protected so that it cannot be altered or deleted without detection, and it must permit recovery and readability of the data and its associated metadata throughout the retention period.¹⁵ Readability throughout the retention period is the requirement that catches teams out, because a proprietary log format from a bot platform you no longer license is not readable in any practical sense five years later.

Settle four points before the bot is switched off:

• What must be retained. Execution logs, audit trails, the outputs themselves, and the configuration in force during the retention window. Include enough of the bot’s definition to explain how a given output was produced, since an audit trail without the logic that generated it answers only half the question.
• In what format. Convert to an open, readable format at the point of archiving rather than assuming the platform will still be available. The conversion needs verification that content and meaning survived intact.
• Under what access controls. The archive should reproduce the access model of the source system, so that records remain available to the people entitled to see them and protected from everyone else.
• For how long, and who confirms disposal. Record the retention end date and the owner responsible for confirming disposal at the end of it. Orphaned archives with no owner are their own compliance problem.

Closing the change control

The change control that retires a validated bot is not closed when the bot stops running. It closes when every one of the following is documented and verified: the replacement is qualified for its intended use, parallel running met its predefined exit criteria, differences found during parallel running were investigated and dispositioned, records from the retired bot are archived in a readable form with retention assigned, the bot’s user accounts and scheduled tasks are disabled so it cannot restart, affected procedures and training are updated, the validated system inventory reflects the retirement, and any interface or downstream system that depended on the bot has been confirmed unaffected.

The account and scheduler item deserves its own line because it is the one most often missed. A retired bot whose service account remains active and whose overnight schedule remains enabled is one restart away from running again against live systems, outside the validated state, with nobody watching. Disable it explicitly and record that you did.

Sequencing the Program Across Eighteen Months

The sequence matters more than the pace. Programs that start with the agent build and work backward to the triage produce a small number of expensive replacements and leave the estate largely untouched. Programs that start with the sort produce a much larger reduction with much less effort.

Months one to three: inventory and sort

Establish the complete inventory, including bots built outside IT that do not appear in the platform console. Ask finance, quality, and site operations directly, because the shadow estate is real and it is usually the least documented part. For each bot capture owner, systems touched, GxP relevance, run frequency, failure history, and change history. Then run the four-way sort with the decision forum described earlier. The deliverable is a signed disposition per bot with recorded rationale.

Months two to six: execute the easy wins

Start retirements and native functionality replacements immediately, in parallel with the tail of the sort. These need no new technology, no model evaluation, and no new validation approach. They reduce the estate, cut the maintenance burden, and build credibility for the harder work that follows. Most programs can materially shrink the estate in this window without touching an agent.

Months four to nine: recover requirements for the agent candidates

Apply the five-step recovery method to the bucket four bots only. This is deliberate sequencing: the recovery work is effortful, and there is no reason to spend it on bots that are being deleted. Expect some candidates to move to other buckets once their rules are visible, which is a good outcome rather than a planning failure.

Months seven to fourteen: build, qualify, and run in parallel

Take two or three agent candidates first, chosen for a mix of real benefit and manageable GxP risk. Build the validation package described earlier. Run the parallel period properly, with predefined exit criteria and a real investigation of differences. Resist the pressure to run five at once, because the first one is where you learn what your acceptance criteria and equivalence tiers actually need to look like, and that learning should be applied to the rest rather than repeated four times.

Months twelve to eighteen: decommission and institutionalize

Close out the retirements formally, complete the archiving, and update the validated system inventory. Then write down the operating model that prevents the estate rebuilding itself: who approves a new automation, what the intake assessment asks, how the deterministic and non-deterministic paths are decided, and what documentation is required before anything runs against a GxP system. Without that last step you will do this again in five years, with agents instead of bots.

Conclusion

The pitch that agentic AI is the natural successor to robotic process automation is appealing because it turns an awkward maintenance problem into a modernization story. It also skips the two questions that decide whether the effort is worth anything. Should this work exist at all, and does the replacement rest on the same validation basis as the thing it replaces? Answer those first and the program gets smaller, cheaper, and considerably easier to defend. Skip them and you spend a year rebuilding automation for processes that should have been retired, while importing non-determinism into places that never needed it.

The honest version of this migration looks like a large number of retirements, a healthy number of bots replaced by functionality the platform vendor has already built and validated, a set of stable bots deliberately left alone with the decision documented, and a small, carefully chosen group of processes where an agent handles exceptions and unstructured input that a rule-based bot never handled well. That last group is where the interesting engineering sits, and it deserves a validation approach built for a system whose path varies, not a copy of the package written for a bot whose path never did.

Sakara Digital works with pharma and biotech organizations facing exactly this decision: an aging automation estate, a board asking about agentic AI, and a quality function that needs to know what the validated state will look like afterward. If you are weighing what to retire, what to leave alone, and what genuinely warrants an agent, and you want an independent read before the program is scoped, we are happy to have that conversation.

For Further Reading