Why Standard Vendor Questionnaires Miss AI Risk

The vendor risk management templates most pharma sponsors use today were built for a different threat model. They ask about SOC 2 Type II attestations, HIPAA safeguards, business continuity plans, data center certifications, penetration test cadence, and change management practices. Those questions still matter. But they were designed around the assumption that a vendor’s software is deterministic, its behavior is documented in a validated specification, and its output today will match its output six months from now for the same input. AI-enabled services break every one of those assumptions.

Consider what has changed at the typical AI-enabled pharma services vendor in the last eighteen months. A CRO that used to code ICSRs with rules-based automation is now using large language models to draft narratives. A safety database provider has embedded a generative feature that summarizes case series. A medical writing vendor uses retrieval-augmented generation to draft first-pass clinical overviews. A regulatory publishing vendor uses AI to auto-classify documents into CTD sections and predict missing cross-references.15 A pharmacovigilance vendor is using agentic AI to triage cases in under a minute where a human reviewer takes thirty.10 None of that shows up in the SOC 2 report. Most of it does not show up in the quality agreement either.

Third-party risk management analysts have described this shift bluntly: vendor risk in 2026 has moved from an annual questionnaire to a continuous discipline, in large part because AI has added a new layer of vendor exposure through the models, data, and features that suppliers embed.1 The problem is compounded by sub-processor opacity. When a vendor’s AI feature is powered by a foundation model from a large model provider, that provider is effectively a sub-service organization whose controls become part of your regulated workflow, whether or not anyone has formally acknowledged it.3

30%
Share of breaches now attributed to third-party involvement, up sharply as AI vendor dependencies multiply1
80%+
Enterprises expected to have used generative AI APIs or deployed generative AI-enabled applications by 202611
$492M
Forecast spend on AI governance platforms in 2026, projected to exceed $1B by 203012

What this means for the pharma sponsor is that the vendor risk program cannot rest on procurement’s annual refresh. The AI features at your vendors are being released, updated, and quietly rewired more often than your vendor risk cycle. A questionnaire administered in January is stale by March. The controls you inspected against last year may no longer describe how your vendor’s platform actually processes your data today. Standard questionnaires miss AI risk not because the questions are wrong, but because the questions do not exist.

The core gap. Traditional vendor questionnaires ask whether a vendor has controls in place. They rarely ask what the vendor’s AI features actually do with sponsor data, which foundation model provider sits underneath, whether the model was retrained last quarter, or how the vendor would notify you if its model behavior materially shifted. Every one of those questions has to be added explicitly.

Five AI-Specific Risk Categories in Pharma Services

Before you can update your questionnaire or your contract templates, you need a shared vocabulary for the risks that are actually new. In our work with pharma sponsors, we have found it useful to organize AI vendor risk into five categories that map cleanly onto the questions procurement, quality, and IT need to ask together. These categories are not exhaustive, but they cover the material risks that show up in inspection findings and in post-incident reviews.

1. Training Data Provenance

The first category concerns the data your vendor’s AI was trained or fine-tuned on. When a CRO’s model was trained on clinical narratives from prior sponsors, the risk is not just about privacy. It is about whether the model has absorbed biases, terminology conventions, or safety patterns from populations that do not reflect your program. Analysts have consistently flagged training data provenance as a top area where vendor questionnaires need to be extended, noting that a vendor who cannot document the origin, licensing status, and de-identification treatment of training data should score materially lower than a vendor with full data lineage.6 Regulators have echoed the point: the EMA reflection paper on AI in the medicinal product lifecycle highlights data provenance, transparency, bias control, and traceable documentation as central to responsible AI use, and expects sponsors to be able to demonstrate those attributes even when they rely on vendor models.4

2. Model Outputs Used in Regulated Deliverables

The second category is about where AI outputs land. There is a large practical difference between a vendor using AI internally to speed up its own workflow, and a vendor’s AI producing text that ends up in your ICSR narrative, your CSR, your Module 2.7, your labeling copy, or your quality decision record. FDA has been clear that when AI informs regulated decisions, including labeling, dosing, safety, batch release, or quality status, the entire system is subject to device-level quality and lifecycle controls, and existing quality agreements often have gaps with respect to AI risks in manufacturing monitoring and control that may lead to challenges during inspections.2 The April 2026 warning letter reinforced that AI-generated outputs or recommendations must undergo review and approval by an authorized Quality Unit representative, and that a human-in-the-loop remains the expectation for GxP functions with direct product quality or patient safety impact.9

3. Sub-Processor AI Usage

The third category is the one that most surprises procurement teams. Your vendor of record is rarely running its own foundation model. Under the hood, it is calling an API to a large model provider, which may itself be routing traffic to different model versions, regions, or infrastructure providers depending on load, jurisdiction, or feature flag. From a SOC 2 lens, that model provider is functionally a sub-service organization, and its controls become part of the story you tell about how your sponsor data is handled.3 From a GxP lens, if your vendor cannot tell you which foundation model handled a given case, you cannot reconstruct how a regulated output was produced.

4. Model Change Control and Version Drift

The fourth category is what happens when the model changes. Traditional software vendors give you release notes. AI-enabled vendors often silently upgrade the model underneath a stable API, and the behavior your validation package was built against no longer describes the behavior of the system in production. Effective mitigation requires explicit change control procedures, version-locked templates, vendor notification service levels, and independent QA audits of automation logic, and service level agreements should cover retention of precision, cadence of retraining, and notification timelines for any change that affects a regulated workflow.8

5. Retention and Downstream Use of Sponsor Data

The fifth category is the most contract-heavy. Many AI vendor agreements, especially those inherited from the SaaS era, contain language permitting the vendor to retain data after contract termination or to use de-identified data for model development without meaningful restriction.7 For a pharma sponsor, that is not a theoretical exposure. It means your protocol text, your adverse event narratives, your commercially sensitive strategy documents, or your patient data may end up as training material for a downstream product used by a competitor. Contracts need to address permissible use, prohibited secondary use, sub-processor compliance, and post-termination obligations with unusual specificity.7

CATEGORY 1

Training Data Provenance

Origin, licensing, de-identification, representativeness, and bias posture of the data used to train or fine-tune the vendor model.

CATEGORY 2

Regulated-Deliverable Impact

Whether and how AI outputs enter GxP records, submissions, safety filings, labeling, or quality decisions the sponsor is accountable for.

CATEGORY 3

Sub-Processor AI Usage

Foundation model providers, inference regions, and any upstream AI services the vendor invisibly relies on for the feature you consume.

CATEGORY 4

Model Change Control

How model updates, prompt changes, and behavior drift are detected, communicated, versioned, and rolled back if needed.

CATEGORY 5

Sponsor Data Retention

What happens to your data during the engagement, after termination, and whether it can be used to train models used for anyone else.

CROSS-CUT

Human Oversight Model

Where humans review AI output, at what sampling rate, with what qualifications, and how those checks are documented for inspection.

Vendor Questionnaire Additions for AI-Enabled Services

The next step is translating those categories into questions your vendor risk team can actually administer. Rather than throwing out your existing questionnaire, which likely covers SOC 2, HIPAA, business continuity, and standard information security well, add a dedicated AI module that maps to the five categories above. The module should be short enough that vendors will actually answer it accurately and specific enough that you can score answers meaningfully. In practice, we recommend twenty to thirty questions, weighted by criticality and applied only to services where AI touches sponsor data or regulated output.

The questions below are drawn from what regulators, industry analysts, and inspection findings suggest matter most. Adjust the wording to your organization, but do not skip the intent behind any of the categories. If your vendor cannot answer a training data provenance question, that is itself a finding worth documenting.

Category Question What a Strong Answer Looks Like
Training Data Provenance What data sources were used to train or fine-tune the models that power the service delivered to us? Please describe licensing, consent basis, and de-identification treatment. Named datasets, documented licensing, de-identification method described, no reliance on scraped or ambiguously sourced data.
Training Data Provenance Was our sponsor data (past or present) used to train, fine-tune, or improve the model? If yes, under what contractual basis and with what protections? Clear yes/no. If yes, contract clause referenced, opt-out available, aggregation approach documented.
Regulated Deliverable Impact Where do model outputs from your service enter GxP records or regulated deliverables (e.g., ICSR narratives, CSR sections, labeling, batch decisions)? Specific list of outputs, mapping to sponsor deliverables, aware of GxP classification implications.
Regulated Deliverable Impact What is your validation and computer system validation approach for the AI features embedded in your service? Please share your validation summary. Validation package aligned to intended use, context-of-use documented, ongoing performance monitoring in place.
Sub-Processor AI Usage List every foundation model provider, API provider, or upstream AI service your platform uses. Include geographic region and any regional failover. Complete list, named providers, documented regions, contractual flow-down to sub-processors.
Sub-Processor AI Usage What contractual provisions do you have in place with those sub-processors regarding data retention, training use, and breach notification? Written flow-down clauses cited, no-training provisions confirmed, breach timelines aligned to sponsor requirements.
Model Change Control How are model updates versioned and communicated? What is the notification window for material changes affecting our workflow? Explicit versioning, defined notification SLA, ability to pin a version for regulated use.
Model Change Control How do you detect and quantify model drift or performance degradation? Please share your monitoring approach and any drift thresholds. Documented monitoring cadence, defined thresholds, remediation triggers, sponsor visibility into metrics.
Sponsor Data Retention How is our data segregated? What are your retention timelines during and after contract termination? Can you produce a certified deletion attestation on request? Logical or physical isolation, defined retention windows, deletion attestation process exists.
Human Oversight Describe your human-in-the-loop model for AI outputs that inform regulated deliverables. Include sampling rate, reviewer qualifications, and documentation approach. Human review defined by risk tier, credentialed reviewers, audit trail of every override or acceptance.

You will notice that none of these questions is theoretical. Each maps directly to something a regulator, a plaintiff’s attorney, or an inspector could ask about. The 2026 SOC 2 auditor guidance now explicitly treats AI model providers as subservice organizations under CC9.2 when they materially affect service delivery, and vendors who cannot answer sub-processor questions are increasingly failing to close enterprise deals.3

Watch for reflexive reassurance. A vendor who responds to every AI question with “we are SOC 2 Type II certified” is telling you they have not thought about the question. SOC 2 does not describe AI-specific controls unless the auditor and the vendor have explicitly scoped them in. Ask for the AI control section of the report; if there is not one, ask why.

Contract Clause Library for AI Provisions

Questionnaires tell you where the risk is. Contracts are how you allocate and remediate that risk. Most pharma vendor contracts, especially older master service agreements and their associated quality agreements, do not contain AI-specific provisions. Adding them retroactively is easier when you start from a library of clauses your legal, quality, and procurement teams have already vetted. The clauses below are intended as a starting library. They should be customized to your risk appetite, your regulator relationships, and the specific service being procured, but they cover the material AI risks a pharma sponsor should reflect in a written agreement.

Data Use and Training

The single most important clause. It must state, unambiguously, that sponsor data (including derivatives, embeddings, and cached prompts) may not be used to train, fine-tune, evaluate, or otherwise improve any model that is used for any other customer, and that sub-processors must be bound to the same restriction. Absent this clause, healthcare AI vendor contracts have been observed to include permissive language allowing retention of patient data after termination or use of de-identified data for model development without restriction.7 Belt-and-suspenders: include an audit right to verify compliance.

Sub-Processor Disclosure and Consent

A clause requiring the vendor to maintain a current list of AI sub-processors (foundation model providers, inference infrastructure providers, upstream AI services), to notify the sponsor of material changes with a defined notice period, and to give the sponsor a reasonable objection right. Model provider sub-processor relationships are one of the most under-examined areas of vendor security posture, and the vendor’s security is only as strong as the provider underneath.6

Model Change Notification

A clause requiring the vendor to notify the sponsor of any material change to the model powering the delivered service, including foundation model version upgrades, fine-tuning updates, prompt template changes, or retrieval-augmented generation index rebuilds that meaningfully affect output. The notice period should be tied to the criticality of the workflow, and the sponsor should retain the right to pin a validated version for GxP use for a defined transition window.8

Drift Monitoring and Performance SLA

A clause requiring the vendor to monitor model performance against defined metrics, report drift beyond agreed thresholds, and cooperate with sponsor-led sample audits. Drift is particularly insidious in pharmacovigilance and other regulated AI applications because a slow one or two percent monthly degradation is invisible in short cycles but compounds into a serious quality problem across a year, and sample-based independent re-review is a proven detection method.5

Regulatory Cooperation and Inspection Support

A clause obliging the vendor to support sponsor responses to regulatory inspections and inquiries related to AI features, including making validation packages, drift monitoring records, and change logs available on reasonable notice. Under ICH E6(R3), sponsors cannot delegate accountability for trial quality or participant safety even when operations are outsourced, and inspection readiness has to extend into the vendor.13

Human Oversight and Quality Unit Review

A clause specifying where human review is required in the AI-augmented workflow and confirming that quality-critical outputs are reviewed and approved by qualified personnel. FDA has made clear that AI-generated outputs or recommendations must undergo review and approval by an authorized Quality Unit representative for many GxP functions.9

Termination and Data Return

A clause governing return, portability, and certified destruction of sponsor data on termination, explicitly covering any embeddings, cached prompts, model fine-tuning artifacts, or retrieval indexes that contain sponsor data. The clause should require written attestation and, at the sponsor’s option, an independent audit of destruction.

Indemnification and Liability

An indemnification clause tailored to AI-specific harms, including intellectual property claims arising from training data, regulatory penalties tied to vendor AI failures, and losses from confidential data leakage through sub-processor AI systems. Traditional SaaS liability caps often exclude precisely the harms that AI failures now create, and negotiation of AI-specific carve-outs has become a common point of contention.14

The SD perspective. Contract clauses are only useful if procurement, quality, and legal have agreed on what “material change” and “material performance degradation” mean in your specific context. We usually recommend that pharma sponsors set those thresholds once, at the program level, and reuse them across every AI-enabled vendor. Otherwise you end up negotiating the same definitions in every deal and getting inconsistent thresholds vendor to vendor, which makes portfolio monitoring impossible.

Ongoing Monitoring for Model Drift and Behavioral Change

The point of a robust questionnaire and a strong contract is to earn the right to monitor. In a pre-AI world, monitoring meant reviewing the annual SOC 2 report and the vendor’s audit findings, and calling for a mid-year check-in if something looked off. That cadence is not enough for AI-enabled vendors, where behavior can shift between annual reviews with no notice and no obvious symptom until an inspector or a plaintiff finds it.

The good news is that the pharmacovigilance community has already done much of the intellectual work here. Drift is defined as performance degradation over time as the data distribution shifts away from training conditions, and detection typically uses continuous monitoring supplemented by sampling agent-processed cases monthly and having experts re-review them.5 The same pattern generalizes to other AI-augmented services: define a metric that matters, sample outputs at a rate calibrated to risk, re-review with qualified experts, and compare results to baseline.

What to Monitor and How Often

The sample rate and monitoring cadence should reflect the risk tier of the vendor. High-risk vendors, whose outputs enter regulatory submissions or safety filings, warrant monthly sampling with formal expert re-review and quarterly sponsor-side quality reviews. Medium-risk vendors, whose outputs inform internal decisions but do not go directly into regulated deliverables, warrant quarterly sampling. Lower-risk vendors can be monitored at a lighter cadence but should still be subject to periodic tests. Continuous monitoring platforms and AI-specific TRiSM tooling can automate a large fraction of this work.12

Signals That Something Has Changed

  • Output length or tone drift. An unexplained shift in the average length, verbosity, or tone of vendor-produced text is often the first visible sign of a model swap or prompt template change underneath a stable API.
  • Confidence score distribution changes. If your vendor exposes confidence scores or probability estimates, a change in the distribution over a short window is a leading indicator of retraining or model swap.
  • New error modes. Errors your vendor’s system did not previously produce, especially on inputs that used to be handled correctly, indicate a behavior shift that may not have been disclosed.
  • Latency or region changes. Requests routed to a new region or with materially different latency may indicate a change in the underlying model provider or infrastructure that has data residency and privacy implications.

A practical monitoring rhythm. One large-molecule sponsor we work with monitors its AI-enabled PV vendor by pulling a random sample of 250 processed cases per month, having two independent medical reviewers re-code them, and comparing results to the vendor’s outputs. A drift beyond 3% coding disagreement triggers a formal vendor meeting; beyond 5% triggers a written escalation and a mandatory review of any regulatory filings drawn from the affected period. The rhythm is boring, which is exactly the point.

AI-Augmented Vendor Risk Assessment Template

The template below is a scored assessment structure we use with sponsors who are extending their vendor risk program to cover AI-enabled services. It is deliberately compact, because a template that takes two days per vendor will not be used. The goal is to produce a defensible, reproducible risk rating that maps to onboarding decisions, contract terms, and monitoring cadence.

1

Scope and Classify the Service

Define the specific service being procured, the sponsor deliverables it touches, and its GxP classification. Determine whether AI features are optional, opt-in, or always-on. Flag any output that enters a regulated deliverable.

2

Administer the Standard Questionnaire

Your existing SOC 2, HIPAA, business continuity, and information security questionnaire. This is the baseline; it should not go away. Score using your current methodology.

3

Administer the AI Module

Deploy the ten-question AI module covering training data provenance, regulated-deliverable impact, sub-processor AI usage, model change control, sponsor data retention, and human oversight. Score each question 0-3.

4

Map Answers to the NIST AI RMF

Cross-walk vendor responses to the GOVERN, MAP, MEASURE, and MANAGE functions of the NIST AI Risk Management Framework, and flag any function with no vendor evidence. GOVERN 6.1 specifically covers third-party AI risk and expects documented controls.16

5

Assign a Risk Tier and Monitoring Cadence

Combine the standard questionnaire score and the AI module score into a composite risk tier (Tier 1 High, Tier 2 Medium, Tier 3 Low). Attach a monitoring cadence and reassessment date to each tier.

6

Confirm Contract Coverage

Cross-check each AI-specific risk against the contract clause library. Where a required clause is missing, either negotiate it in or record a formal risk acceptance signed at an appropriate level of authority.

7

Baseline Performance and Establish Drift Metrics

Before the vendor goes live, establish baseline metrics you will monitor for drift: precision, recall, coding agreement, output length distribution, or whatever is most meaningful for the service. Document thresholds.

8

Schedule the Next Review

Because AI vendors change more frequently than traditional vendors, the reassessment cycle should be shorter. High-tier vendors reassessed every six months, medium-tier annually, lower-tier every eighteen months at most.

The output of the template is a one-page vendor risk profile that procurement, quality, IT, and legal can all read in the same terms. That single-page profile becomes the artifact you bring into the vendor governance committee, into the contract negotiation, and eventually into the regulatory inspection binder. It also becomes the reference point when something changes: when the vendor announces a new AI feature, when a sub-processor changes, when drift is detected, or when a new regulation lands. Everyone knows what the baseline was and can reason about the deviation.

Operationalizing the Program Across the Vendor Portfolio

Rolling out an AI vendor risk program across an established pharma sponsor portfolio takes deliberate sequencing. The mistake we see most often is treating every vendor equally on day one, which stalls the effort in the sheer volume of assessments. Instead, sequence the work by risk exposure.

Start with the vendors whose AI outputs most directly touch GxP deliverables: safety databases and PV vendors, medical writing services, regulatory publishing partners, and any CRO delivering AI-assisted clinical narratives or coding. These vendors carry the most inspection exposure and the most compressed remediation timelines when something goes wrong. Move next to vendors whose AI supports commercial and medical affairs functions: MLR review acceleration, real-world evidence platforms, market access modeling. Save vendors whose AI is purely operational (IT service desk automation, invoice processing, general-purpose meeting summarization) for the final wave.

Sequencing by regulatory adjacency. The right first question is not “which vendor is largest?” or “which contract is up for renewal?” It is “which vendor’s AI output could show up in a regulatory submission, a safety filing, or a Quality Unit decision?” That is where inspection exposure lives, and that is where you want the program to be strongest first.

Operationally, the program needs a home. In most pharma sponsors, that home turns out to be a cross-functional AI vendor governance forum that meets monthly and includes representation from Procurement, IT Security, Quality (with GxP owners as needed), Regulatory Affairs, Legal, and Privacy. The forum owns the AI vendor inventory, the risk-tier assignments, the drift monitoring dashboard, and the roadmap for extending the program to new vendor categories. It also owns the escalation path when a vendor discloses a material AI change or when internal monitoring detects drift.

The program also has to interlock with your broader AI governance function. Deloitte and other analysts have consistently observed that AI risk management should be integrated with the organization’s broader risk approach, and teams working on AI need training in the elements of that risk, including cyber, fraud, and intellectual property, while continuous monitoring identifies errors and biases that were not apparent in training or that develop with changing environments.14 A pharma sponsor’s vendor AI risk program should feed into and draw from its internal AI governance policy, not run as a parallel silo.

Finally, the EU AI Act’s provider-deployer construct is worth mapping onto your vendor portfolio, even if your primary regulatory exposure is FDA. When your vendor is the provider of a high-risk AI system and you are the deployer, obligations for quality management, documentation, logging, monitoring, and human oversight are split between you along explicit lines, and the alignment window before full enforcement is closing.17 Pharma sponsors that operate transatlantically will want the vendor risk program to reflect that dual regulatory reality now, not later.

Conclusion

Vendor risk management for AI-enabled pharma services is not a bolt-on. It is a deliberate extension of the sponsor’s existing quality and risk framework into a domain where the vendor’s product is now partly a model, the model’s behavior can change silently, and the regulatory expectation of sponsor accountability has not moved an inch. The organizations that navigate this well are not the ones with the longest questionnaires. They are the ones who have built a small, coherent AI vendor risk program that classifies vendors by regulatory adjacency, updates contracts with a defensible clause library, monitors for drift on a rhythm that matches the underlying risk, and treats sub-processor AI usage as first-class information rather than a footnote.

Sakara Digital works with pharma and biotech organizations building exactly this kind of AI vendor risk program: questionnaire modules that ask the right questions of CROs, safety databases, publishing vendors, and medical writing partners; contract libraries that hold up under scrutiny; monitoring rhythms that catch drift before an inspector does; and governance forums that keep procurement, quality, IT, and legal working from the same page. If you are extending your third-party risk framework to cover AI-enabled services and want an independent perspective on where to start, we are happy to have that conversation.