Vendor Validation Packages for AI and SaaS Tools: What You Can Reuse and What You Must Still Test

What Is in a Vendor Validation Package

Most life sciences buyers of SaaS now hear the same sales pitch. The vendor explains that its platform is “validated” or “GxP ready,” and then offers a validation package, sometimes free and sometimes as a paid add-on, that is meant to shorten the customer’s own validation effort. For AI tools, the package often adds a model description, accuracy figures, and a statement about how the vendor tests each model version.

The contents vary from vendor to vendor, but a typical package includes some or all of the following:

• A product requirements specification or functional specification describing what the standard product does.
• Test scripts and executed test results for the standard product, often for each major release.
• A traceability matrix linking product requirements to the vendor’s tests.
• A validation summary report or “certificate of validation” for the release.
• Independent reports and certificates: a SOC 2 Type 2 report, an ISO/IEC 27001 certificate, sometimes ISO 9001, and for AI vendors increasingly ISO/IEC 42001.
• Release notes and a change log for each release.
• Infrastructure qualification records for the hosting environment, or the cloud provider’s own compliance documents.
• Template documents for the customer: a sample user requirements specification, a sample validation plan, and sample test scripts the customer can run.
• For AI tools: a model card or model description, a performance summary on the vendor’s test data, and a description of how model versions are managed.

Why the Package Exists

The package exists because the market asked for it. Under the older computer system validation model, each customer of a multi-tenant SaaS product wrote and ran its own scripts against the same standard functions. Each customer tested the same login screen, the same audit trail, and the same electronic signature workflow. A vendor that has already tested those functions under its own quality system can save each customer that effort, and the regulators have said that this kind of reuse is acceptable when it is justified.

The Question Every Package Raises

The useful question is not “Is this package good?” It is “What does this package prove, and about what?” A vendor’s executed test of its standard approval workflow proves that the standard workflow behaved as designed in the vendor’s test environment on the date of the test, for that release. It does not prove that your configured approval workflow, with your roles, your routing rules, and your integration to your document system, does what your procedure requires. Both statements matter. Only the first one can come from the vendor.

Once a team separates “evidence about the product” from “evidence about our use,” most of the confusion about vendor packages goes away. The rest of this article works through that split.

What Regulators Say About Using Supplier Evidence

There is no single rule on vendor validation packages, but the main texts point the same way: you may rely on supplier work, you must assess it first, and you keep the accountability.

FDA Computer Software Assurance

FDA’s guidance Computer Software Assurance for Production and Quality Management System Software is final. The current version is dated February 2026 and states that it supersedes the final guidance issued on September 24, 2025.¹² Computer Software Assurance (CSA) is FDA’s risk-based approach to establishing confidence that software used in production or the quality system is fit for its intended use. The guidance was written by FDA’s device and biologics centers for medical device production and quality management system software, in consultation with the drug center. Pharma and biotech companies do not fall under its scope directly, but many use its reasoning because it is FDA’s clearest current statement on risk-based assurance of commercial software.

On supplier evidence, the guidance is direct. It lists, among the things that give flexibility, “validation activities performed by other entities (e.g., developers, suppliers, cloud service providers).”¹ Under purchasing controls, it says a manufacturer could “incorporate the software development practices, validation work, and electronic information already performed by developers of the software as the starting point and determine what additional activities may be needed.”¹ Note the words “starting point.” The guidance also accepts that a regulated company may have limited access to vendor information, and it notes that an onsite audit may not be feasible. It lists other ways to assess a vendor: reviewing SOC reports and ISO certifications, reviewing the vendor’s development, quality, and cybersecurity practices, and reviewing the software’s data integrity controls.

The guidance’s SaaS example is especially useful. It describes a cloud product lifecycle management system used with standard configuration only. The manufacturer assesses the vendor, puts a service agreement in place covering security, data integrity, privacy, availability, change management, and business continuity, and then handles updates this way: “The SaaS vendor provides the manufacturer documentation summarizing the changes, testing, and testing results of all automatic updates made to the SaaS system functions identified by the manufacturer as part of the service agreement.”¹ The manufacturer then assesses each change against its intended use and performs risk-based testing scaled to the impact. That one example contains most of this article: reuse the vendor’s testing, assess it against your use, test what your use requires, and write the duty to provide the evidence into the agreement.

The Pharma Regulations Underneath

For drug manufacturers, the legal requirements come from the CGMP regulations and Part 11. 21 CFR 211.68 requires that computers and related systems used in drug manufacturing be “routinely calibrated, inspected, or checked according to a written program designed to assure proper performance.”³ 21 CFR 11.10(a) requires “Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records.”⁴ Neither text says who must perform the testing. Both place the obligation on the regulated company. FDA’s 2003 Part 11 guidance adds the risk-based basis: “We recommend that you base your approach on a justified and documented risk assessment and a determination of the potential of the system to affect product quality and safety, and record integrity.”⁵

For clinical systems, FDA’s 2024 final guidance on electronic systems in clinical investigations is the most specific FDA text on vendor validation evidence. It says that when an IT service provider performs validation, the regulated entity may consider reviewing the provider’s documentation on system development, validation processes, functional testing, and change control.⁶ In the same answer it sets the boundary: “Validation should be applied to system functionality, configurations specific to the clinical trial protocol, customizations, data transfers, and interfaces between systems (e.g., interoperability and communication).”⁶ The vendor can cover standard functionality. Protocol-specific configuration, customizations, data transfers, and interfaces belong to the sponsor’s use.

GAMP 5 Second Edition

ISPE’s GAMP 5 Second Edition, published in July 2022, is the main industry framework for computerized system compliance. ISPE describes the edition as encouraging regulated companies to “maximize supplier involvement to leverage knowledge, experience, and documentation where possible.”⁷ The guide has a full chapter on supplier activities and sections titled “Leveraging Supplier Input” and “Leveraging Existing Information,” along with appendices on supplier assessment, design review and traceability, testing, data migration, and AI and machine learning.⁸ ISPE’s own summary of the edition notes that it addresses validation planning and reporting for SaaS solutions and the assessment of cloud service providers, which move some risk management activities outside the regulated company.⁹

The GAMP Good Practice Guide on testing, in a third edition published in July 2026, makes the same point from the testing side. ISPE says that “Suppliers are encouraged to strengthen their testing approaches, allowing regulated companies to leverage supplier testing more effectively,” with the aim of adopting system updates quickly and safely.¹⁰

EU GMP Annex 11 and PIC/S

Annex 11 (2011) is still the version in force in the EU. As of October 2026, the European Commission’s EudraLex Volume 4 page lists Annex 11 with its January 2011 revision.¹² Three of its clauses apply directly to vendor packages. Clause 3.3: “Documentation supplied with commercial off-the-shelf products should be reviewed by regulated users to check that user requirements are fulfilled.” Clause 3.4: “Quality system and audit information relating to suppliers or developers of software and implemented systems should be made available to inspectors on request.” Clause 4.5 requires the regulated user to take all reasonable steps to ensure the system was developed under an appropriate quality management system and says the supplier should be assessed appropriately.¹¹

The draft revision of Annex 11, released for consultation in 2025 and not yet final, goes further. Section 7.2 says that when relying on a vendor’s qualification, the regulated user should, by risk and criticality, audit or thoroughly assess the vendor’s procedures and documentation and “the potential to leverage these rather than repeating the activities.”¹³ Section 9.9 says qualification and validation documentation may come from a vendor in part or in whole, but “the regulated user is fully accountable and should carefully review and authorise the use of the documentation,” including whether it covers the implemented version.¹³ Because this text is a draft, treat it as a clear signal of where EU inspectors are heading, not as a requirement in force.

PIC/S guidance PI 011-3 on computerized systems draws the same line in one sentence. In describing supplier test scripts, it says: “The test scripts related to the user requirements specification are the responsibility of the regulated users.”¹⁴

The Reuse Map: Artifact by Artifact

A package is not one thing to accept or reject. It is a set of artifacts, and each one proves something different. The table below is the map we use with clients: for each common artifact, what it can support, what it cannot, and how to use it in your own validation record.

Vendor Test Evidence

Executed vendor test results are the most valuable part of any package and the part most often used wrongly. They are valuable because they can cover the standard, unconfigured behavior of the product in more depth than most customers would ever test: boundary conditions, error handling, audit trail entries, and signature manifestations. They are used wrongly when a team cites them as proof for a function it has configured in a different way.

The test for reuse is whether the vendor’s test exercised the same behavior you will rely on. If you use the standard audit trail as delivered, the vendor’s test of that audit trail is direct evidence. If you have changed which fields are audited, or turned on a reason-for-change prompt the vendor tested as off, the vendor’s test proves less, and you need at least a short confirmation test of your setting. Annex 11 clause 3.3 frames this as a review against user requirements, and that is how the record should read: requirement, vendor evidence reviewed, conclusion, and any added testing.¹¹

Independent Reports and Certificates

SOC 2 reports and ISO certificates are evidence about the vendor’s organization, not about a release. FDA’s CSA guidance lists them as inputs to a vendor assessment, alongside review of development and quality practices.¹ ISO/IEC 27001 is a standard for information security management systems; ISO/IEC 42001 is a standard for AI management systems.¹⁶¹⁷ A certificate against either one tells you the vendor runs a management system that an auditor has checked, for the scope written on the certificate. It does not tell you whether the build you are about to use was tested against your requirements. The details of reading a SOC 2 report for GxP purposes are covered in a separate SD article linked at the end of this piece, so we will not repeat them here.

Release Notes and Change Records

Release notes are the artifact you will use most often after go-live, because they drive every release assessment. Their quality varies widely. Useful release notes name each changed function, say whether the change alters existing behavior or adds new behavior, say whether the new behavior is on or off by default, and point to the vendor’s testing of the change. Weak release notes describe new features in marketing terms and say nothing about changes to existing behavior. Part of judging a package is judging the release notes the vendor publishes, which we return to below.

Traceability Matrices

A vendor’s traceability matrix proves that the vendor’s own requirements are covered by the vendor’s own tests. That is a sign of a disciplined development process, and it is useful. But it only checks the vendor’s requirements against the vendor’s tests. Your inspector will ask a different question: show me how each of your requirements was verified. The working approach is a single matrix of your own requirements, where each line points either to a vendor test (with the reason it is enough) or to one of your tests. That keeps the vendor matrix as supporting evidence without letting it stand in for yours.

What You Must Still Own and Test

The second half of the split is the work that no vendor can do for you, however good its package is. These items come from the way you use the system, so the evidence has to come from you.

Intended Use and Your Own User Requirements

Every framework above starts from intended use. FDA’s CSA guidance asks manufacturers to identify the intended use of the software and of its individual features and functions before deciding what assurance is needed.¹ A vendor can describe what its product can do. Only you can say which of those capabilities you will rely on for a GxP decision, which records the system will hold, and what a failure would mean for patients, product, or data.

The same is true of user requirements. Many packages include a sample user requirements specification. It is a useful draft. But a vendor-written requirements document describes the product, so every requirement in it is met by definition. The draft Annex 11 addresses this case: for purchased or SaaS systems, a requirements specification “may be provided by the vendor,” but the regulated user should review and approve it, decide whether the system meets GMP needs and company processes as is, and take ownership of the document for the implemented version.¹³

Configuration

Configuration is where most SaaS validation risk lives. Roles and permissions, workflow routing, required fields, picklist values, signature meanings, notification rules, and data retention settings are all set by the customer, and each one can change the behavior of a standard function. FDA’s clinical systems guidance names “configurations specific to the clinical trial protocol” as something validation should cover.⁶ The draft Annex 11 asks that the chosen configuration be documented in a controlled configuration specification.¹³

The practical approach is to record every configured setting that touches a GxP requirement, rate each by risk, and test the higher-risk ones yourself. Low-risk settings can often be verified by inspection of the configuration record rather than by a test script. Under a CSA approach, scaling the effort to risk this way is the intent.

Integrations and Interfaces

The vendor tests its API. It does not test your middleware, your field mappings, your error handling, or the system on the other end. Integration testing almost always belongs to the regulated company, and it is often the highest-risk part of a SaaS implementation because data moves between systems with different rules. Annex 11 clause 5 asks that systems exchanging data electronically include appropriate built-in checks for correct and secure entry and processing of data.¹¹ Your tests should show that records arrive complete, that rejected records are flagged rather than dropped, and that a failure in one system does not create a silent gap in the other.

Data Migration

When legacy records move into the new system, the migration is yours. Annex 11 clause 4.8 is explicit: “If data are transferred to another data format or system, validation should include checks that data are not altered in value and/or meaning during this migration process.”¹¹ A vendor may provide a migration tool, and its testing of the tool can be reused for the tool’s general behavior. The mapping of your legacy fields, the handling of your historical audit trails, and the reconciliation of record counts and values all depend on your data, and the evidence must come from your migration runs.

Procedures, Training, and Records

A validated system run by untrained staff under procedures that do not match the configuration is not in a state of control. Your SOPs, your training records, your access review process, your backup and restore checks for any data you keep outside the platform, and your periodic review are all part of the validated state. None of them is in a vendor package.

AI Tools: Performance on Your Own Data

AI tools add one item to the “must own” list that matters more than any other: how the model performs on your data, in your process. A vendor’s accuracy figure is a result on the vendor’s test set. Your records may differ in format, vocabulary, quality, language, document type, or the mix of easy and hard cases. A model that scored well on the vendor’s data can score worse on yours, and the vendor package cannot tell you by how much.

Why Vendor Accuracy Numbers Do Not Transfer

The best known example comes from healthcare rather than pharma, but the lesson carries over. In 2021, researchers at Michigan Medicine published an external validation of a widely deployed proprietary sepsis prediction model in JAMA Internal Medicine. Across 38,455 hospitalizations, the model’s hospitalization-level area under the curve was 0.63, which the authors described as “substantially worse than the performance reported by its developer.”¹⁸ The paper compares this with figures of 0.76 to 0.83 that the developer reported in internal documentation. The model did not identify 67 percent of patients with sepsis while alerting on 18 percent of all hospitalized patients.¹⁸

The authors said the large difference in reported results was likely due to how their study handled the timing of sepsis onset. Evaluation choices of that kind are exactly what a local test controls. A local evaluation uses local definitions, local data, and local workflows, and those are what decide whether a tool is fit for your use. A 2021 article in the New England Journal of Medicine, which used the same sepsis model as an example, described the general problem: “Dataset shift occurs when a machine-learning system underperforms because of a mismatch between the data set with which it was developed and the data on which it is deployed.”¹⁹

What the Draft Annex 22 Asks For

The EU’s draft Annex 22 on artificial intelligence, released for consultation alongside the draft Annex 11 and not yet final, is the most detailed regulatory text on testing AI models in GMP. Its scope is narrow: static, deterministic machine learning models used in critical GMP applications. It says generative AI and large language models should not be used in critical GMP applications, and that in non-critical uses a qualified person should review outputs.²⁰ Even if your tool falls outside its scope, its test design principles are a sound reference.

Three points bear directly on vendor packages. First, section 2.2 says documentation should be available and reviewed by the regulated user “irrespective of whether a model is trained, validated and tested in-house or whether it is provided by a supplier or service provider.”²⁰ Second, the intended use, the subgroups of input data, and the acceptance criteria are to be defined by a process subject matter expert and approved before acceptance testing starts. Third, section 5.1 says “Test data should be representative of and expand the full sample space of the intended use.”²⁰ A vendor’s test set was built for the vendor’s idea of the intended use. Only you can build a test set that represents yours.

A Local Performance Test in Five Steps

What to Ask the Vendor So You Can Test

The vendor package for an AI tool should make your local test possible, not replace it. Ask for a non-production environment where you can run your own test set against the production model version, a stable model version identifier that appears in logs and outputs, advance notice of model changes, and the ability to re-run your test set on a new version before it reaches production. A vendor that cannot provide those things is telling you that you will not be able to keep the tool in a validated state, whatever its package says. The separate SD articles on AI vendor qualification and on change control for prompt and model updates cover the vendor questions and the change process in more depth.

How to Judge the Quality of a Package

Not all vendor packages are worth reusing. Some are thorough records of controlled testing. Others are slide decks with screenshots. Before you cite any of it in your validation record, review it as you would review an internal test package, and record that review. The ISPE SaaS article puts it this way: “The regulated company needs to evaluate the SaaS provider’s documentation package in view of their own risks.”¹⁵

We use six checks.

Use the CSA Record Elements as a Checklist

FDA’s CSA guidance lists what an assurance record should contain: the intended use of the feature, the result of the risk-based analysis, a description of the testing, issues found, a conclusion on acceptability, who performed the work and when, and review and approval where appropriate.¹ Those elements make a fair test of a vendor’s records too. If a vendor’s executed test cannot tell you who ran it, when, on which version, and what went wrong, it is weak evidence, however many pages it has.

The same guidance favors digital records, such as system logs and audit trails, over screenshots and paper. Vendors that test with automated tools often hold excellent digital evidence in their test management and defect tracking systems and then export only a thin summary into the “package.” Ask what evidence is behind the summary and how you can see it during an audit or an inspection.

Sample, Do Not Repeat

Judging a package does not mean repeating it. A focused review of a sample of executed tests in the high-risk areas you care about (audit trail, electronic signatures, access control, calculations, and any function you will rely on for a GxP decision) gives you a defensible view of the whole. Record which tests you sampled, what you found, and your conclusion on how far you will rely on the rest. If the sample shows problems, widen it or replace that part of the package with your own testing.

Watch for Templates Presented as Evidence

Some packages are mostly templates: a blank validation plan, sample test scripts for the customer to run, and a summary report with spaces to fill in. These can save drafting time, but they are not evidence of anything. Count them as tools, not as validation, and do not let a thick binder of templates create a false sense that the work is done.

Keeping the Package Current With Every Release

A SaaS package describes one release. The vendor will release again, often on a fixed schedule, plus patches. Each release changes the product your evidence describes. A package that is not refreshed with each release becomes a historical record rather than evidence of a current validated state.

Why Updates Need Their Own Assessment

Updates can change controls you depend on, not just add features. In an April 2026 warning letter to a drug manufacturer, FDA cited the firm under 21 CFR 211.68(b) for inadequate access controls over laboratory systems. In its response to FDA’s inspection observations, the firm said that a software update had allowed users to bypass user-security access restrictions and that it had disabled the automatic update function. FDA found the response inadequate and asked for a full assessment of computer system security and validation status.²¹ That case involved on-site laboratory software rather than SaaS, but it shows why a release must be assessed against the controls you rely on before it reaches production.

FDA’s clinical systems guidance makes the general expectation clear: changes to electronic systems, including software upgrades and security and performance patches, “should be evaluated and validated throughout the life cycle of the system depending on risk.”⁶

A Release Assessment Routine

This routine follows the CSA SaaS example closely: the vendor provides documentation of each update, the manufacturer assesses the effect on intended use, tests to the level of impact, and keeps a record.¹ It only works if the vendor provides the inputs in time, which is why the contract matters.

The Preproduction Window

The routine above depends on time between when you see a release and when it reaches your production tenant. The ISPE SaaS article recommends that, where significant customer configuration exists, the vendor give customers a preproduction environment to assess upcoming changes, run regression testing, and train users before release, and that the quality agreement state the time allowed.¹⁵ Without that window, your release assessment becomes an after-the-fact review of a change already in use.

Contract Terms That Keep the Evidence Useful

A vendor’s package is only as dependable as the vendor’s obligation to keep providing it. Sales commitments made during selection are often not written into the agreement, and a validation package that the vendor “usually” provides can stop arriving after an acquisition, a reorganization, or a price change. The terms below belong in the master agreement, the quality agreement, or both.

The draft Annex 11 section 7.5 lists what a contract with a service provider should cover, including the activities and documentation to be provided, conditions for supplier audits, support during regulatory inspections, communication of quality and security issues, an exit strategy that lets the regulated user retain control of its data, and agreement “on the process for release of new system versions and on the regulated user’s possibility to test these prior to release.”¹³ FDA’s clinical systems guidance recommends a written agreement, such as a master service agreement with a service level or quality agreement, covering scope, roles and responsibilities, and access to data throughout the retention period.⁶

Put the Quality Terms in One Place

The ISPE SaaS article recommends that, even where some quality terms appear in service level or master service agreements, the quality agreement should be the one document that holds all agreed quality responsibilities.¹⁵ That helps during an inspection, when you need to show quickly who is responsible for release testing, notification, and evidence.

Negotiate Early

These terms are easiest to win before signature, when the vendor wants the deal. After go-live, the customer has less to offer. Bring quality and validation leads into the selection process early enough that the package review and the contract terms are part of the commercial decision, not a task for after the purchase order.

Match the Terms to Risk

Not every SaaS tool needs every term. A low-risk tool that supports a quality record but does not drive a GxP decision may need only release notes and a notice period. A system that holds batch records, manages deviations, or produces data for a regulatory submission needs the full set. The same risk assessment that sizes your testing should size your contract terms.

Conclusion

Vendor validation packages are worth having. Regulators and industry guidance support reusing supplier evidence, and a well-built package can remove a large share of the repetitive testing that used to fill SaaS validation projects. The discipline that makes reuse safe is the same in every case: separate evidence about the product from evidence about your use, reuse the first after a recorded review, and own and test the second. For AI tools, that second category always includes performance on your own data, because a vendor’s accuracy figure describes the vendor’s test set, not your records. And because SaaS and AI products change with every release, the package is only useful if the contract keeps it current and your team assesses each release against the functions you rely on.

Sakara Digital works with pharma and biotech organizations to plan and run risk-based validation of SaaS and AI tools, including reviewing vendor packages, writing reuse rationales that hold up in inspections, designing local AI performance tests, and setting the quality agreement terms that keep evidence current. If you are weighing a vendor’s validation package and want an independent view of how far you can rely on it, we are happy to have that conversation.

For Further Reading