Building an AI Model Registry: What to Track and Why

Why a Model Registry Matters Now

Three converging pressures make the AI model registry an executive-level concern in 2026 rather than a niche MLOps concern.

First, regulatory registration is arriving. The EU AI Act establishes a public EU database of high-risk AI systems under Article 71, with providers and deployers required to submit registration data described in Annex VIII. The Commission has committed to having the database operational before the AI Act’s application date of 2 August 2026.¹ A pharma company that cannot enumerate its own high-risk AI systems internally will not be able to complete this external registration accurately.

Second, guidance for regulated computerized systems has caught up. GAMP 5 Second Edition, published by ISPE in July 2022, added Appendix D11 to address AI and machine learning. The appendix treats dynamic ML sub-systems as a distinct life cycle inside the broader computerized system life cycle, with specific expectations around data acquisition, training and test datasets, and performance-metric acceptability.² ISPE followed with a standalone GAMP Guide on Artificial Intelligence in mid-2025, extending those principles across concept, project, and operational phases, including model risk management and change control.⁴

Third, health authorities have staked out clear traceability expectations. The EMA’s reflection paper on the use of AI in the medicinal product life cycle, adopted in October 2024, tells applicants that they must ensure “robust models are applied, traceable documentation is established and maintained, and that secondary assessments of development practices are conducted.”³ The FDA, Health Canada, and MHRA jointly published ten Good Machine Learning Practice guiding principles and, for medical-device software, five guiding principles for predetermined change control plans.⁵⁶ In every one of these frameworks the presumption is that the sponsor knows precisely which model is in use, on what version, trained on which data, and monitored against which metrics.

The organizations we see move first tend to share a specific trigger. Either an internal audit surfaced an AI model in a GxP workflow with no validation record attached, or a customer or partner asked for a model card during a due-diligence review and no one could produce one, or the legal team read Article 71 and asked which of the company’s models will need to be registered. Any of those moments creates the mandate. The registry is what turns the mandate into a durable operating capability.

The NIST AI Risk Management Framework, meanwhile, has become the reference architecture that most Quality and IT leaders are aligning to for internal governance. NIST defines an AI system inventory as “an organized database of artifacts relating to an AI system or model, which may include system documentation, incident response plans, data dictionaries, links to implementation software or source code, names and contact information for relevant AI actors, or other information that may be helpful for model or system maintenance and incident response purposes.”¹³ That definition maps almost directly to what a pharma model registry needs to do; the pharma extension is the addition of GxP-specific metadata and the tight coupling to the validated systems inventory and the EDMS.

The Model Registry Versus the Validated Systems Inventory

Before getting into fields and schemas, it helps to be precise about how the AI model registry differs from the validated systems inventory that GxP-regulated pharma companies have maintained for years.

The validated systems inventory is a list of computerized systems. An entry describes a system such as the LIMS, the MES, or the electronic quality management system, with attributes covering GxP classification, GAMP software category, business owner, system owner, validation status, last periodic review date, and the location of the validation dossier. The unit of tracking is the system.

The AI model registry tracks a different unit: the model. A model has a name, a version, a purpose, an intended use, training data, a specific set of performance metrics, and a lifecycle that includes retraining events. Many models live inside systems that already appear in the validated inventory. A signal-detection model that scores adverse events inside the safety database is one model. A demand-forecasting model that runs inside the commercial planning platform is another. A generative model that drafts sections of a clinical study report is a third. None of these are systems in the classic CSV sense. All of them are regulated artifacts that need to be tracked.

The two inventories are related, and the registry should reference the system inventory. But they are not the same list, and treating them as one is a common early mistake. A single computerized system might host multiple models, each with its own lifecycle, and a single model might be deployed across multiple systems (an entity-extraction model that serves both regulatory and medical affairs). The registry needs the flexibility to represent both cases without forcing artificial one-to-one mappings.

A Practical Data Model for the Registry

The right set of fields is the one your validation team, your legal team, and your model owners can all use without re-entering the same information three times. Google’s Model Cards paper established the pattern most organizations now build on: model details, intended use, factors, metrics, evaluation data, training data, quantitative analyses, ethical considerations, and caveats.⁷ The pharma-specific data model extends this with the fields regulators and auditors will ask about.

The table below is a starting point for a mid-cap pharma or biotech registry. It groups fields by section so that different roles (data science, validation, quality, legal) can own their sections without treading on each other.

Two design choices are worth spelling out. First, every field that references another artifact (validation dossier, monitoring dashboard, incident log) is a link, not a copy. The registry is a coordination layer, not a duplicate repository. Second, both the human name and the immutable Model ID are captured. Human names change; Model IDs must not. That distinction is what makes the registry usable ten years from now, when reviewers need to trace a decision back to the exact model version that produced it.

Sample Registry Entries

A schema in the abstract is easier to argue about than a schema with entries in it. Two sample entries are shown below, one for a signal-detection model in pharmacovigilance and one for a generative model used in regulatory writing. These are illustrative composites, not descriptions of any specific product.

Entry A: Adverse Event Triage NLP

Entry B: Regulatory Section Drafter (GenAI)

What these entries do in practice is give a reviewer, an auditor, or a partner a one-page answer to the question “what is this model, who owns it, and how do we know it works?” That is the single most valuable service a registry provides.

Integrating With GxP Validation Records

The registry must plug into the GxP quality system without replacing it. That means the registry does not hold validation evidence; it points to validation evidence. Six integration points matter most.

Version Control, Monitoring, and Decommission

An AI model registry that only tracks live production models is not a registry; it is a landing page. The real value comes from tracking the whole lifecycle, including versions that are no longer running.

Version control

Semantic versioning (MAJOR.MINOR.PATCH) works reasonably well for models if you define the semantics up front. A change in training data, model architecture, or intended use is a MAJOR version bump. A retraining on the same schema with the same architecture and improved data is a MINOR bump. A bug-fix or configuration change is a PATCH bump. MLflow’s model registry provides the mechanics for this out of the box, with immutable model versions and explicit stage transitions (Staging, Production, Archived), plus aliases such as @champion or @challenger for A/B evaluation.⁸ Whether or not you use MLflow specifically, the pattern is the correct one.

The registry entry for each model should show the current production version, the previous production version (still deployed to a subset for A/B, or in cold storage), and the full version history with the change control record ID attached to each transition. That history is what makes reproducibility possible three years later.

Monitoring

Monitoring information belongs in the registry as pointers, not as content. Each entry links to a live dashboard (data drift, prediction drift, output distribution, key performance metrics) and stores the drift threshold that triggers action. When monitoring detects an issue, the resulting incident and its resolution are logged, and the registry entry updates to reflect the current state.

Decommission and retirement history

Models get retired. Sometimes because they are superseded, sometimes because the business process changed, sometimes because a regulator raised a concern, sometimes because a validation review concluded the model no longer meets acceptability criteria. In every case, the registry needs to record the retirement event and preserve enough context to answer questions later.

The reason a full retirement history matters becomes clear during a regulatory inspection or a post-market safety review. An inspector who asks “what version of the triage model classified this case in June 2027?” needs an answer that is unambiguous, immutable, and reachable in minutes. The registry, done well, is what provides that answer.

A related discipline is preservation of the training data itself at the time of retirement. Data used to train an early production model is often the very data that a later challenger asks questions of, or that a regulator wants to see when evaluating whether a retired model was fit for its intended use during the time it was active. The registry should record where that dataset lives, in what form (raw versus feature-engineered), and under what access controls. Where the training data included patient-level information under a specific consent basis, the retirement record should also capture the retention period and the point at which the frozen training snapshot itself must be destroyed. Compliance is not just about live models; it is about the archived tail as well.

Governance: Ownership, RACI, and Access

A registry without an owner becomes a spreadsheet. A registry without governance becomes an incomplete spreadsheet. The organizations that get this right pin down four things before they build.

The single accountable owner

One named executive is accountable for the registry as a whole. In most mid-cap pharma the right home is the Chief Data Officer, the Head of Digital Quality, or the head of the AI Center of Excellence where one exists. Split ownership fails; a single throat-to-choke succeeds.

Per-model ownership

Every entry has a named model owner (typically a data scientist or data science manager), a named business sponsor (VP or equivalent in the function that uses the model), and a named validation lead (from QA/CSV). All three are required fields. An entry with any of these blank cannot progress to production status.

RACI for lifecycle events

The registry codifies who is Responsible, Accountable, Consulted, and Informed for six lifecycle events: onboarding a new model, promoting to production, retraining, monitoring escalation, PCCP-scope changes, and retirement. Below is a starting RACI that fits most mid-cap organizations.

Access and disclosure

The registry is not a public artifact. It contains commercial detail (training data, vendor terms, performance metrics on internal populations) that many companies will not want on the open internet. Standard practice is three access tiers: internal read (broad access across the enterprise), internal edit (per-model owners and validation leads), and external disclosure (model card export for partner due diligence or regulator response). The export view is a filtered subset of the registry entry, produced on demand from the same source data, so that there is no risk of the external and internal descriptions of the same model drifting apart.

Under the EU AI Act, high-risk AI systems will have a mandatory registration in the public EU database, with providers responsible for entering the data listed in Sections A and B of Annex VIII and public authority deployers responsible for the data in Section C.¹ The disclosure view of the registry should be designed so that Annex VIII fields map cleanly to registry fields and can be produced without manual reformatting. Companies that treat this as an ongoing export from a single source of truth, rather than as a separate manual submission, avoid the drift risk that comes with parallel records.

Model card portability

Where possible, use the industry-standard Model Card format for the external export view. Google’s Model Card Toolkit provides both a JSON schema and a rendering library, and it has become the de facto interchange format for exchanging model documentation between organizations.¹⁴ A pharma registry that speaks Model Card natively for external exchange, and stores the pharma-specific extensions internally, gets the best of both worlds. Partners and regulators receive an artifact in a form they already know how to read; internal Quality and Legal teams retain the full metadata they need for audit and inspection response.

Build Versus Buy: Tool Landscape

No single tool on the market today covers the full pharma registry requirement out of the box. Most organizations end up with a two- or three-layer stack: an MLOps tool that handles the technical registry (versioning, artifacts, lineage), a governance layer that handles the pharma-specific metadata (GxP flag, GAMP category, validation dossier link, PCCP), and integration to the EDMS and validated systems inventory for the regulated records themselves.

The core MLOps registries

MLflow. Open source. Provides the model registry primitives: immutable versions, stage transitions, aliases, tags, and lineage back to the run that produced each version. Widely adopted, low licensing risk, well-supported inside Databricks. Requires pharma-specific extensions to hold GxP metadata.⁸

Weights & Biases. Strong on developer experience and visualization; extended with prompt tracking and evaluations for LLM work. Best for research and R&D use cases, less naturally suited to the audit-focused stance a GxP registry needs.⁹

Neptune. Positioned as the enterprise metadata store, with strong lineage tracking connecting models to exact data versions, code commits, and configurations. Explicitly targets regulated industries requiring detailed audit trails.⁹

Databricks Unity Catalog. Unified governance for data and AI inside the Databricks platform. Automatic lineage across ingestion, ETL, dashboards, and ML models, with attribute-based access control and audit logging. Attractive for organizations already standardized on Databricks; less useful in polyglot environments where a lot of the AI work happens outside the platform.¹⁰

Life-sciences-specific patterns

Two industry examples are worth studying, though neither is a general-purpose registry product.

MELLODDY. The Machine Learning Ledger Orchestration for Drug Discovery project brought together ten pharma companies plus technical partners to demonstrate cross-pharma federated learning at scale, with a shared platform for training QSAR models on 2.6 billion confidential activity data points across 21 million molecules while keeping each company’s data private.¹¹ MELLODDY’s operational model — versioned shared training rounds, per-partner model artifacts, and clear ownership of the global model versus the local models — is a useful reference for organizations thinking about federated AI, where the registry needs to distinguish between the model as trained collectively and the model as deployed locally.

OHDSI ATLAS. ATLAS is a web-based tool for designing and executing analyses on standardized observational data in the OMOP Common Data Model, used across an OHDSI network of more than 331 data sources with 2.1 billion patient records across 34 countries.¹² ATLAS is not a general model registry, but it is an important pattern for tracking cohort definitions, phenotypes, and prediction models with shared governance across many sites. Organizations running real-world evidence or observational research programs should study how ATLAS handles versioning and cross-site reproducibility.

A 90-Day Implementation Roadmap

The most common failure mode for pharma model registry programs is scope. Teams try to design a perfect registry that captures every conceivable field, and eighteen months later they still do not have one. A better pattern is to build a minimum viable registry, populate it, get it into the periodic review calendar, and then evolve.

Conclusion

The AI model registry is not another checkbox. It is the operational spine that connects intended use, training data, validation, monitoring, and decommission for every AI system the organization runs. Companies that stand it up early get three benefits at once. They can answer the “which models do we run and how do we know they work?” question in minutes rather than months, they walk into the EU AI Act Article 71 registration exercise with the data they need already in hand, and they give their Quality and Legal teams a shared operating tool that keeps AI governance close to the business rather than at arm’s length from it.¹³

Sakara Digital works with pharma and biotech organizations building this kind of AI governance capability, from initial model inventory through registry design, GxP integration, and readiness for EU AI Act registration. If you are exploring what a model registry should look like at your organization and want an independent perspective on where to start, we are happy to have that conversation.