Migrating AI Models Between Vendors: The Validation and Data Integrity Questions

Why a Vendor Switch Is Not a Version Upgrade

Our earlier article on change control for prompt and model updates dealt with a model that changes under a tool while the provider stays the same: a new dated version, a retirement notice, a sealed regression set run on both versions. A move to a different vendor shares some of that work, but it differs in three ways that matter for validation and data integrity.

First, far more of the tool’s behavior moves at once. The new model was trained by a different team, on different data, with different conventions for prompts, parameters, tool calls, and output formats. Second, the new provider brings its own retention rules, processing locations, lifecycle policy, and contract. Third, the old provider still holds whatever state your tool left behind: uploaded files, stored conversations, vector stores, fine-tuning data, batch results. A version upgrade inside one vendor touches none of those last two areas. A vendor switch touches all of them.

Four Common Migration Paths

“Changing vendors” covers several different moves. They share a checklist, but the risks concentrate in different places.

Forced Moves and Chosen Moves

A chosen move has a timeline you set. A forced move has a timeline the vendor sets, and the published commitments vary widely. Anthropic states that it gives at least 60 days of notice before retiring a publicly released model; its deprecation history shows Claude Sonnet 4.5 users were notified on September 30, 2026, of a November 30, 2026, retirement on the Claude API.¹ OpenAI commits to at least six months of notice for generally available models, at least three months for specialized variants, and much shorter notice, such as two weeks, for preview models.³ Amazon Bedrock gives models launched before September 7, 2026, a Legacy period of at least six months, and gives models launched on or after that date a Legacy period of either six months or 45 days, shown on the model card. It also states that its dates can differ from the model provider’s own dates.⁴ Microsoft Foundry sets an 18-month lifecycle for most generally available models, 12 months for models from Anthropic, DeepSeek, Fireworks, and Mistral AI, sends retirement notices at least 60 days ahead, and does not extend retirement dates.⁵ Google lists most Gemini models as available for at least 12 months after release, and gives at least 45 days to migrate once it posts a retirement date for its shorter-term models; its lifecycle table, for example, lists October 20, 2026, as the retirement date for gemini-2.5-pro.⁶

Sixty days is not much time to qualify a new supplier, run a paired comparison, rework prompts, update work instructions, and get quality approval. For most quality teams the practical question is not whether a model or vendor change will happen to a validated AI tool. It is whether the evidence and the records will be ready when it does.

Classify the Change Before You Test Anything

Most change control procedures have a category for a major change to a validated system and a separate process for bringing on a new supplier. A vendor switch is both, and the two processes need to run together rather than one after the other. If the supplier assessment starts only after the technical testing is complete, a finding about data processing locations or retention can send the team back to the beginning.

The first decision to make and record is whether this is a change to an existing validated system or a new system. If the intended use, the users, the input types, and the human review step stay the same, treating the switch as a major change usually makes sense, because the existing requirements and test set still describe what the tool must do. If the switch comes with new functions, such as agent tools, new document types, or a larger context window that changes how inputs are split up, the requirements need revisiting and the work is closer to a new validation. Write the rationale into the change record either way. An inspector will ask why the team chose one route over the other.

FDA’s January 2025 draft guidance on AI used to support regulatory decisions for drugs and biological products, which is still a draft, frames AI credibility around a specific context of use and proposes a risk-based credibility assessment framework.⁷ Whatever weight a company gives that draft, the logic carries over: credibility evidence belongs to a model in a context of use. Replace the model and the evidence has to be re-established for that context of use, even if nothing else about the process changes.

Six Migrations Happening at Once

It helps to split the switch into its parts and assign each one an owner and an evidence package. Teams that treat the switch as one test event usually finish the model comparison and then discover the other five parts during cutover week.

The Supplier Changes Even When the Model Name Does Not

The path most often under-classified is the move of the same model to a different platform. Anthropic’s documentation states that its retirement dates apply to Anthropic-operated platforms, and that partner-operated platforms such as Amazon Bedrock and Google-managed Claude models set their own retirement schedules.¹ Its data retention page adds that on Bedrock and the Google-managed offering, the cloud provider is the data processor, not Anthropic.² Amazon states that for Bedrock usage, only the dates on the Bedrock model card apply.⁴

So a tool that moves the same Claude model from one platform to another has a new supplier, a new data processor, and a new retirement calendar. EU GMP Annex 11 expects formal agreements with third parties, including service providers, that state the third party’s responsibilities clearly, and it makes the competence and reliability of the supplier a key factor in selection.⁸ That applies to the platform you are moving to, regardless of whose name is on the model.

Who Signs Off

A vendor switch needs more signatures than a model version change. The business process owner confirms the intended use is still met. The system owner confirms the configuration and interfaces. Quality approves the validation evidence and the supplier assessment. Privacy and legal review the processing locations and data terms, because the new provider may process prompts in different countries. IT security reviews keys, network paths, and access. Name these roles in the change record at the start so the approval route is known before the deadline gets close.

Re-Establishing Intended-Use Performance on the New Model

Fitness for Use, Not Sameness

The natural instinct is to test whether the new model gives the same answers as the old one. That is the wrong target. The old model’s outputs are not ground truth. They include its errors, and an equivalence test would treat a fix as a failure and preserve the old model’s weaknesses as the standard.

The acceptance criteria should come from the intended use, the same way they did for the original validation: the expected result for each test case, defined by a qualified person, with criteria by risk stratum written before the run. The old model’s results are still useful, but as a comparison point. They tell you where behavior moved, which is where reviewers and procedures need attention. They do not tell you what correct looks like.

Replay Unchanged, Then Adapt

Microsoft’s model migration guidance gives a sequence that fits validated work well. Freeze the test inputs, expected results, and success criteria for the whole migration. Run the current configuration on the target model without changes first, so you can separate what the model changed from what your later edits change. Score the current model on the frozen set to set a baseline, then score the target on the same set.⁹ The same guidance makes a point many teams learn too late: capture of production content is opt-in and, in Microsoft’s words, “never retroactive,” so traffic you did not record cannot be replayed later.⁹

For a GxP tool, captured production inputs often contain confidential data and sometimes personal data. Decide in advance how representative inputs will be collected, de-identified where needed, approved for test use, and stored. That decision belongs in the validation plan, not in a developer’s laptop folder.

Read the Comparison Case by Case

An average pass rate hides what matters. Two models with the same overall score can fail on entirely different cases, and the cases where the new model fails and the old one passed are the ones that can reach a batch record, a safety case, or a submission. Sort every test case into one of the cells below and report the counts by risk stratum.

The regression cell gets the most scrutiny. Each case needs an explanation: a prompt that relied on the old model’s habits, a formatting change that broke parsing, a real capability gap. Some can be fixed inside the change. Some cannot, and the decision then is whether stronger human review covers the risk or the switch has to wait.

What Usually Shifts Between Vendors

Microsoft’s guidance lists the behaviors that commonly change when a workload moves to a new model: verbosity, reasoning depth, adherence to structured output, the shape of tool calls, refusal behavior, and latency variance.⁹ Research points the same way. The authors of PromptBridge, a 2025 study of cross-model prompt transfer, describe prompts tuned for one model losing accuracy when reused on another, a problem they call model drifting and report as common and severe across many model configurations.¹⁰ A separate 2025 paper on prompt migration, based on an enterprise search application, argues for treating prompt migration as a structured process and reports that it recovered the reliability lost when the underlying model changed.¹¹

Even a single named service can move. Chen, Zaharia, and Zou compared the March and June 2023 versions of GPT-3.5 and GPT-4 and found large shifts on the same tasks; GPT-4’s accuracy at telling prime from composite numbers fell from 84% to 51% between the two versions.¹² If two dated versions of one service can differ that much, a model from a different vendor should be assumed to behave differently until the test set shows otherwise.

Reviewers Learned the Old Model’s Mistakes

In most GxP uses of generative AI, a human reviews the output before it is used. That review is a control, and it is tuned to the model it has been reviewing. Over months, reviewers learn where the old model tends to go wrong: a particular kind of table it misreads, a phrase it overuses, a section it tends to skip. A new model makes different mistakes in different places, and a reviewer watching for the old patterns can miss the new ones.

Use the comparison results to train reviewers on the new failure modes before go-live. Update the work instruction where the review steps assumed the old model’s behavior. Raise the review sampling rate, or move from sampled to full review, for a defined period after cutover, and record when and why it returns to normal. This is the part of a vendor switch that most often goes unrecorded, and it is the part an inspector is most likely to ask about after an error.

Repeat Runs and Nondeterminism on New Infrastructure

Moving to a new serving stack changes run-to-run behavior even when the weights do not change. A September 2025 analysis by Thinking Machines Lab sampled 1,000 completions of the same prompt from an open-weight model at temperature 0 and got 80 unique completions; with inference kernels designed to give the same result regardless of batch size, all 1,000 were identical.¹³ The authors trace most of the variation to batch size changing with server load, which a customer of a shared API does not control.

Two consequences follow. A move from self-hosted to an API, or from one inference server to another, needs repeat runs of each test case, with acceptance criteria for stability as well as correctness. And an on-premises to cloud move with identical weights still needs a comparison, because the hardware and serving software that produce the output have changed.

What Does Not Carry Over: Parameters, Fine-Tuning, and Embeddings

A validated AI tool’s configuration baseline lists everything that shapes its output. In a vendor switch, much of that baseline has to be rebuilt rather than copied, and each item that changes needs a reason in the change record.

Parameters That Stop Existing

Some configuration settings that teams recorded as controls may not exist on the new model. Anthropic’s deprecation page shows that the sampling parameters temperature, top_p, and top_k were removed in API version 2026-10-21 and now return an error; earlier API versions also return an error when they are set to a non-default value on Claude 4.7 and later models.¹ Microsoft’s migration guidance makes the general point that temperature, top_p, maximum token, and reasoning-effort settings do not map one-to-one across model generations, and that newer families may not support some of them.⁹

This matters because many validation packages list “temperature set to 0” as a control for consistent output. If the parameter no longer exists, the control is gone, and the risk assessment needs a replacement: repeat-run stability testing, tighter output constraints, more human review, or a combination. Record the replacement explicitly. A baseline that still says “temperature 0” for a model that cannot accept it is a documentation error waiting to be found.

Structured Output, Tool Definitions, and Calling Code

Prompts are the visible part of a migration, but Microsoft’s guidance points to four other surfaces that usually change: parameters, tool definitions, output schemas, and the calling code, including differences in API shape, streaming format, and field names that downstream parsing depends on.⁹ For a validated tool, the downstream parsing is often where a defect hides. A field that changes name can leave a value empty in the receiving system without raising an error. Interface tests that confirm every field the tool writes downstream are part of the migration, not an optional extra.

Token limits and tokenizers also differ between vendors. The same document produces a different token count on a different model, so any logic that splits long documents into chunks, truncates long inputs, or stops at a length limit will behave differently. Test with the longest realistic inputs, not only typical ones, and confirm what the tool does when an input exceeds the new limit.

Fine-Tuned Models Start Over

A fine-tuned model does not move between vendors. What you keep is your training data, your training records, and your evaluation results. Microsoft states that fine-tuned workloads cannot be auto-upgraded and that adapting them means re-training or distillation against the replacement base model.⁹ Amazon notes that once a foundation model enters the Legacy state on Bedrock, no new fine-tuning jobs can be created on it.⁴

For validation, re-tuning on a new base model is a new training event, with its own data provenance, training record, and evaluation. If the training dataset was assembled years ago and never placed under version control, a forced migration is when that gap becomes visible. Keeping the training data versioned and under your own control is the single most useful step a team can take before it ever needs to switch.

Embeddings and the Retrieval Index

Tools that answer questions from a document collection usually store each document chunk as an embedding, a list of numbers produced by an embedding model. Vectors from different embedding models cannot be compared with each other, so changing the embedding model means rebuilding the whole index. A peer-reviewed 2025 ACL paper on converting between embedding models describes this re-embedding of entire datasets as computationally expensive and notes it comes with no guarantee of better performance.¹⁴

The rebuilt index is a new validated artifact. The evidence should show that the document count matches the source, that chunking settings and metadata carried over, that every vector records the embedding model that produced it, and that a retrieval test set of known questions returns the right documents at an agreed rate. Do not run a mixed index with some vectors from the old model and some from the new one; similarity scores across the two are meaningless.

Data Residency and the Historical Record

Where the New Vendor Processes Prompts

Processing location is set by contract and by configuration, and both can change in a vendor switch. Microsoft’s documentation explains that for Global deployment types, inference may be processed in any Azure region; Data Zone types process data within a defined zone (United States, European Union, or Asia Pacific); and Standard and Regional Provisioned types process prompts and responses within the chosen Azure geography. Data at rest stays in the designated geography in all cases, and Microsoft states it can add regions to a data zone without prior notice.¹⁵ OpenAI’s data controls page lists the United States, Europe, and the United Arab Emirates as regions with both storage and processing, and several other countries, including the United Kingdom, Japan, and Canada, with regional storage only.¹⁶

There is a timing trap here for forced migrations. Microsoft states that new models launch first in Global Standard, then reach Data Zone deployment types, and reach single-geography deployment types last.⁵ A replacement model may therefore be available only with global processing when the old model’s retirement date arrives. If your privacy assessment or clinical consent language assumed processing in one region, the replacement may not be usable in the configuration you need, and you need to know that on the day the notice arrives, not the week before retirement.

What the Old Vendor Still Holds

Before the switch, list everything the tool stored with the old vendor. The answer depends on which features the tool used, and the retention rules differ by feature. OpenAI keeps abuse monitoring logs for up to 30 days; stores Responses API state for 30 days by default; keeps assistants, threads, and vector stores until they are deleted, removing them 30 days after deletion and retaining them indefinitely if they are never deleted; and keeps files until deleted.¹⁶ Anthropic’s documentation states that prompts and outputs are not retained by default, except for designated models that require 30-day retention, while files uploaded through its Files API are kept until deleted or until a set expiration, and batch processing data is retained for 29 days.²

Each stored object needs a decision: migrate it, archive it, or delete it. Vector stores and uploaded reference files at the old vendor may hold copies of controlled documents. Stored threads may hold the only copy of an interaction if the tool was built to use them as its history. Fine-tuning files may hold training data that has to be retained under your own records rules.

The Vendor Is Not Your Records System

If a migration reveals that the only record of past outputs is in the old vendor’s stored conversations, the gap existed well before the migration. Under 21 CFR 211.180, required records or copies must be readily available for authorized inspection throughout the retention period, and may be kept as original records or as true copies.¹⁷ Annex 11 adds that archived data should be checked for accessibility, readability, and integrity.⁸ A vendor’s default retention, which can be 30 days or none at all, meets neither expectation.

The inference record for a GxP AI tool belongs in a system you control: the input, the output, the exact model identifier, the vendor and platform, the configuration version, the timestamp, the user, and the reviewer’s decision. Our article on decommissioning AI systems covers what that record needs to contain over a long retention period. In a vendor switch, the question is narrower: is every element of that record already in your system, and if not, what has to be pulled from the old vendor before access ends?

Exit Steps for Vendor-Held Data

Keeping the Audit Trail Continuous Across the Cutover

An investigator looking at an output from 18 months ago needs to know which model produced it, through which vendor, under which configuration, and who accepted it. A vendor switch is where that chain most often breaks, usually because the log format changes and nobody maps the old fields to the new ones.

Annex 11 Clause 4.8 Applies to Inference Logs

Annex 11 states: “If data are transferred to another data format or system, validation should include checks that data are not altered in value and/or meaning during this migration process.”⁸ Teams apply that clause to batch records and laboratory data without hesitation, and then move an AI tool’s inference logs into a new schema with no checks at all. If historical logs move from the old application tables, or from a vendor-side store, into a new structure, the migration needs the same checks: record counts, field-level comparison on a sample, and confirmation that coded values mean the same thing after the move.

Annex 11 also requires that audit trails “need to be available and convertible to a generally intelligible form and regularly reviewed.”⁸ An audit trail that can be read only inside the old vendor’s console, or only by the engineer who wrote the old logging code, does not meet that after the switch.

Map the Log Schema Field by Field

Mark the Cutover in the Record

Record the exact date and time the old model stopped producing GxP records and the new one started. If the cutover is staged, the record must show which model served each request. Microsoft’s guidance notes that real migrations are often split, with part of a workload on the new model and part on the old one, sometimes for weeks.⁹ That is a reasonable way to reduce risk, as long as each logged request carries the model and vendor that served it. A log that says “the model” without saying which one cannot support an investigation during that period.

If the team runs the new model in shadow mode, processing real inputs without its outputs being used, keep the shadow outputs clearly separated from the record. They are test evidence, not GxP records, and mixing the two creates confusion about which output was relied on.

Monitoring Baselines Reset

Trend charts for error rates, refusal rates, reviewer override rates, and processing times all break at the cutover. The old values describe a different model. Start a new baseline on the cutover date, show the break on the chart, and note it in the next periodic review. Comparing the new model’s first month against the old model’s long-run average, without that note, invites the wrong conclusion in either direction.

Lock-In and the Contract Terms That Decide the Next Switch

Our earlier article on the hidden cost of AI vendor lock-in looked at lock-in as a strategic and financial risk. Here the focus is narrower: the terms that decide how much time and evidence you will have at the next switch, because a company that has migrated once will migrate again.

Compare the Published Lifecycle Policies

These are public policies as published in October 2026, and providers revise them. Check them at contract signature, record the version you relied on in the supplier file, and check again at each periodic review.

Clauses Worth Asking For

Public policies are the floor. For a model version that supports a validated tool, ask for more, and ask during selection, when you have the most negotiating power.

• Longer notice for named versions. A notice period longer than the public policy for the specific model versions your validated tools use, sized to your real validation timeline.
• Continued access through the notice period. A commitment that existing validated deployments keep access until the retirement date, with no inactivity cut-off and no loss of access in test or disaster recovery environments.
• Notice of changes behind the same identifier. Notification of infrastructure or safety-filter changes that can alter output under an unchanged model ID.
• Export of stored objects. Export of all stored files, vector stores, conversations, batch results, and fine-tuning data on request and at termination, in a documented format.
• Deletion with confirmation. Deletion on request, a stated removal period, and written confirmation.
• Processing location. Commitments on where prompts are processed and stored, and notice before that changes.
• Subprocessors. Notice of new subprocessors that handle prompts or outputs.
• Transition help. Reasonable help during a switch, including continued access to the old model for a defined overlap.

Anthropic published a commitment in November 2025 to preserve the weights of its publicly released models for at least the lifetime of the company.¹⁸ That is useful for research and accountability, but it does not commit to customer access after retirement, so it should not appear in a validation plan as a fallback.

What the EU Data Act Adds

For companies operating in the EU, the Data Act has applied since September 12, 2025, and sets a framework for customers switching between data processing services.¹⁹ The European Commission’s explainer states that switching charges, including data egress charges, are removed entirely from January 12, 2027, and that providers of Platform and Software as a Service must, at minimum, export data in a commonly used, machine-readable format.²⁰ Whether a particular AI service falls within those rules is a question for legal counsel, but the Act gives buyers a stronger position when negotiating export and transition terms.

Build for the Next Migration

The best preparation for a vendor switch happens at the first implementation. Keep the inference record, the test set, the prompt repository, and the training data in systems you control. Route model calls through a thin internal layer so that the vendor-specific code is in one place. Record the embedding model on every vector. Avoid putting vendor-side stored conversations or vendor-hosted vector stores in the record path, or accept that choice explicitly and write the exit burden into the risk assessment.

A Migration Checklist

The checklist below follows the order most switches take. Adapt it to your change control procedure, and drop items that do not apply to your tool, with a note explaining why.

Conclusion

Moving a validated AI tool to a new vendor is not a test event with paperwork attached. It is a group of related migrations, each with its own evidence: the model’s behavior against the intended use, a rebuilt configuration baseline, a re-indexed document collection where embeddings change, a continuous record and audit trail, a new supplier file, and reviewers who know how the new model fails. The teams that handle it well are the ones that kept the inference record, the test set, and the training data in their own hands from the start, so that a retirement notice starts a planned change rather than a scramble. The acceptance question is the same one asked at the original validation, whether the tool is fit for its intended use, and the record has to show the answer for every output on either side of the cutover.

Sakara Digital works with pharma and biotech organizations on the validation, data integrity, and supplier questions that come with AI tools in regulated work. If you are facing a model retirement or planning a move between providers and want an independent view on where the evidence gaps are, we are happy to have that conversation.

For Further Reading