Analytics Engineering in Regulated Environments: dbt, Version Control, and Validation

Where the Collision Actually Happens, and Where It Does Not

The story most quality leaders hear is that a data team has started building the reporting layer in dbt, that everything now lives in a git repository, and that the old validation package does not fit. The specifics vary. The reaction is consistent: this looks new, therefore it looks risky.

It is worth separating the parts of that reaction that are correct from the parts that are habit. Validation expectations in pharma have never actually required a Word document. They require that a system used for a regulated purpose is shown to be fit for its intended use, that changes are controlled, that records are attributable and durable, and that someone competent reviewed the work. The document was the medium, not the requirement. FDA’s Computer Software Assurance guidance for production and quality management system software, finalized in September 2025, makes the point explicitly by asking for assurance activities scaled to risk and for records that capture what was actually done, rather than for a fixed set of paperwork applied uniformly.¹²

GAMP 5 Second Edition, published by ISPE in 2022, moved in the same direction. It updated the original guide to accommodate iterative and incremental development, continuous delivery, and modern service models, and it treats the linear waterfall lifecycle as one option rather than the default.¹⁴ If a validation approach for analytics engineering feels like it has no precedent, that is usually a sign that the organization is still working from the 2008 mental model rather than from current guidance.

The three things that genuinely change

Three shifts are real, and they are the ones worth spending governance attention on.

The unit of change gets much smaller and much more frequent. A traditional validated reporting system might change four times a year through formal change control. A transformation repository might change four times a week. The controls have to work at that cadence or people will route around them. This is the single most common failure. An organization keeps a change control process designed for quarterly releases, applies it to a weekly pipeline, and within two months half the changes are being made directly in the warehouse outside the repository, which destroys the evidence trail the repository was supposed to provide.

The boundary of the system becomes harder to draw. A validated LIMS has an obvious perimeter. A warehouse with 600 models feeding dashboards, notebooks, regulatory extracts, and a commercial forecast has no natural perimeter at all. Scope has to be asserted deliberately, model by model, and enforced mechanically. That is the subject of a later section and it is where most of the real work sits.

Evidence becomes generated rather than composed. In a document-driven approach a person writes down what was tested and attaches a screenshot. In a code-driven approach the build produces a record of what ran, against which commit, in which environment, with which result. That is a better record, but only if someone deliberately keeps it. Build logs that expire after 90 days are not a validation record.

Why Version-Controlled Transformation Code Is Better Evidence

Start from what the spreadsheet-based reporting layer actually looked like in most mid-size pharma and biotech organizations, because that is the honest comparison. An analyst pulled an extract, applied logic in a workbook, saved a copy with a date in the filename, and mailed it. If a number was later challenged, reconstructing how it was produced meant finding the right version of the file, opening the formulas, and asking the analyst what they remembered. Change control existed on paper. In practice the record of the transformation was the file itself, and the file could be edited without leaving a trace.

Against that baseline, a transformation repository is not a compliance risk. It is a substantial improvement, and the improvement lines up closely with the data integrity attributes regulators already use.

The ALCOA+ mapping

ALCOA+ is the shorthand for the data integrity attributes set out in the MHRA GxP data integrity guidance and reinforced in PIC/S PI 041-1: attributable, legible, contemporaneous, original, and accurate, plus complete, consistent, enduring, and available.⁴⁵ Applied to the change record for a transformation, a well-run repository maps unusually well.

Read that table honestly and the picture is clear. Five of the nine attributes are satisfied better by a repository than by any document-based process. Three need a specific control added. One, endurance, needs a records decision that has nothing to do with the tool.

Why this fits how FDA now thinks about assurance

The CSA guidance asks organizations to think about the intended use of the software, the risk if it fails, and then to select assurance activities proportionate to that risk, keeping records that capture the activity and the result.¹³ A transformation repository with continuous integration produces exactly that kind of record, and it produces it automatically for every change rather than only for the changes someone remembered to document.

The same logic runs through the current expectations for computerized systems more broadly. The 2011 EU GMP Annex 11 already required change control, audit trails, and evidence of validation appropriate to risk.⁶ The 2025 draft revision, published for consultation in July 2025, expands that from a five-page annex into a nineteen-page document covering supplier oversight, identity and access management, expanded audit trail expectations, and cybersecurity.⁷ The concept paper behind the revision is explicit that the original text no longer reflects how systems are actually built and operated.⁸ A change record that is generated as a side effect of doing the work is a better answer to that direction of travel than a change record assembled after the fact.

The clinical parallel

Anyone who has worked in clinical programming has seen a version of this argument settled already. Statistical programming groups have kept SAS and R programs under version control for years, and the derived dataset is understood to be reproducible from the program plus the source data plus the environment. ICH E6(R3), adopted at Step 4 in January 2025, treats data governance as a shared sponsor and investigator responsibility and puts weight on traceability from source through to the reported result.¹³ FDA’s 2024 questions and answers guidance on electronic systems in clinical investigations takes a similar risk-based line on validating the systems that create and maintain those records.¹² The transformation layer in a warehouse is the same problem in a different department.

The Genuine Gaps: What Version Control Does Not Give You

If the argument stopped at the mapping above it would be a sales pitch rather than an assessment. There are real weaknesses, and quality reviewers find them quickly. Naming them first is more persuasive than being caught by them.

Gap one: history is immutable by convention, not by construction

The claim that git history cannot be altered is not true. Commits are content-addressed, so a commit cannot be edited in place, but the branch pointer can be moved to a rewritten history. Amending a commit, rebasing a branch, and force pushing all replace the recorded sequence with a different one, and the standard tooling documents how to do it.²⁴ With sufficient permission on the hosting platform, someone can rewrite the history of the main branch and the previous version becomes unreachable.

That is a genuine difference from an audit trail in a validated application, where the trail is enforced by the application and not by the operator’s restraint. The fix is straightforward but it has to be deliberate. Branch protection or repository rulesets on the production branch, blocking force pushes and branch deletion, restricting who can bypass, and recording bypass events, converts an assumption into a control.²²²³ The important shift is one of framing: branch protection stops being a developer convenience and becomes a documented GxP control with an owner, a configuration baseline, and a periodic review.

Gap two: authorship is self-asserted

By default the author recorded on a commit comes from local configuration. A user can set it to any name and address. For attribution that has to hold up, commits on protected branches should be cryptographically signed, and the platform should reject unsigned commits.²⁵²³ The signing identity then needs to be tied back to a named individual whose training record and role authorization exist in the quality system. Without that link, the repository shows that someone with a particular key made a change but not that the person was qualified to make it, which is what 21 CFR Part 11 and Annex 11 are both reaching for.⁹⁶

Gap three: timestamps come from the client

Commit timestamps are supplied by the machine that made the commit. They are usually right and they are trivially falsifiable. For contemporaneity that will survive questioning, rely on server-side events: the merge time recorded by the hosting platform, and the start and end times of the continuous integration run. Those are generated by systems the developer does not control. Where the validation record needs a time, use those rather than the commit time.

Gap four: a repository is not a records archive

This is the gap that gets missed most often, and it is the least technical. A repository holds the current history for as long as the organization keeps paying for the platform and nobody deletes it. GxP records have defined retention obligations tied to the product, not to the tooling. The predicate rules set retention against events like batch expiration and distribution, which routinely exceed the life of any particular analytics platform.¹⁰ FDA’s data integrity questions and answers guidance is direct that records must remain accessible and readable throughout the retention period.¹¹

What has to be added is a records decision, not a feature:

• Define what the record is. Usually the tagged release, the resolved project manifest for that release, the test results from the qualification run, and the approval. Not the whole repository.
• Export it to the controlled records system at each release, in a format that will still be readable without the platform. Plain text, structured data files, and PDF renderings of the approvals.
• Assign a retention period consistent with the records the analytics support, and a disposition process.
• Cover deletion risk. An organization-level repository deletion, an expired platform contract, or an account closure can remove the entire history. Backup and restoration have to be procedural, tested, and owned.

Gap five: version control tracks code, not data

The repository proves what the logic was. It does not prove which rows went through it. Two runs of identical code against a source table that changed underneath produce different outputs, and only one of them may match the number in the report. Reproducibility of a regulated output needs the code version, the source data state, and the run record joined together. In practice that means recording, for every production build of an in-scope model, the commit hash, the run timestamps, the source freshness state, the row counts, and the test results, and keeping that run record with the same retention as the output it supports. This is the piece teams most often assume the tool handles for them.

Scoping GxP: Classify Models by What They Feed

The fastest way to make an analytics platform ungovernable is to declare the whole warehouse GxP. Nobody can carry formal qualification across 600 models, so the policy is quietly ignored and the organization ends up with no real control anywhere. The alternative is not to declare the whole warehouse out of scope. It is to classify.

The classification question is not what a model contains. It is what a model feeds. A model holding batch data that only ever lands in an exploratory notebook carries different risk from a model holding the same data that feeds a product quality review. Intended use drives the assurance effort, which is the same logic CSA applies to production and quality system software and the same logic GAMP 5 applies through risk assessment.¹¹⁴

A three-tier scheme that holds up

Marking and enforcing the boundary inside the project

A classification that lives in a separate spreadsheet will drift within a quarter. The classification has to live next to the code and the build has to enforce it. In a dbt project the mechanics are ordinary:

• Declare the tier as model metadata. Every model carries a required key in its configuration, for example a gxp_tier value under meta, alongside a named system owner and the intended use in one sentence. Tags let the same models be selected as a group at build time.²¹
• Fail the build when a model is unclassified. A continuous integration step parses the project manifest and rejects any model without a tier. Unclassified is not a neutral state, it is a defect. This is the check that keeps the scheme alive.
• Enforce the contamination rule mechanically. The same step walks the dependency graph and fails when a Tier A model has an upstream neighbor at a lower tier.
• Declare downstream consumers as exposures. dbt exposures let the project name the dashboards, extracts, and applications that consume a model, which turns “what does this feed” from tribal knowledge into a queryable part of the project.¹⁸ This is what makes the classification defensible rather than asserted.
• Put schema shape under contract for Tier A. Model contracts cause a build to fail when the returned columns, types, or constraints do not match what was declared, which prevents a silent shape change from reaching a regulated consumer.¹⁷
• Route review by path. Code ownership rules require a quality reviewer on changes touching Tier A directories, so the approval requirement is enforced by the platform rather than remembered by the author.

The tiering also solves a political problem. Data teams resist GxP scope because they expect it to slow everything down. When the scheme puts most of the warehouse in Tier C and applies real rigor to a defined minority, the conversation changes from resistance to negotiation about which models belong where. That is a much more productive argument to be having.

Tests Are Not Qualification

This is the point where technically strong teams most often mislead themselves, and it is worth being precise. A passing test suite and a qualification are different things, and the difference is not rigor. It is purpose.

Three kinds of automated check, doing three different jobs

Modern transformation frameworks give at least three distinct mechanisms, and conflating them muddles the evidence.

• Data tests run after a model is built and assess the condition of the data that came out. Uniqueness, non-null, accepted values, referential relationships, and custom assertions.¹⁵ They answer “is today’s data acceptable.” They are a monitoring control, not a proof of logic.
• Unit tests validate transformation logic against small static inputs before the model is materialized, and dbt Labs recommends running them in development and continuous integration rather than in production because the inputs never change.¹⁶ They answer “does the logic do what we specified.” That is much closer to what qualification needs.
• Contracts enforce the declared shape of the output at build time and stop the model building when it does not conform.¹⁷ They answer “is the interface still what consumers were promised.”

A team that only writes data tests has good data monitoring and no evidence about its logic. A team that only writes unit tests has evidence about logic and no monitoring of production data condition. For a Tier A model, a reviewer should expect all three, and should expect the team to be able to say which question each one answers.

What qualification adds

Qualification demonstrates fitness for intended use. That requires several things a test suite does not provide on its own.

Where the requirement lives

Requirements do not need to live in a separate document, and they usually should not, because a separate document drifts. They can live in the repository as structured files with stable identifiers, reviewed through the same process as the code. What matters is that a requirement has an identifier, that tests reference it, and that a traceability view can be generated on demand showing which requirements are covered and by what. Generated traceability is more reliable than a hand-maintained matrix, because it cannot silently fall out of date. It is also the artifact that most impresses an auditor who arrived expecting to be handed a stale spreadsheet.

Environments, Data, and the Promotion Path

Separation of development, test, and production is one of the oldest expectations in computerized system compliance and one of the most frequently misdemonstrated. In an analytics platform the separation is often real but poorly evidenced, because it is implemented through naming conventions that a reviewer cannot verify.

Demonstrating separation, not just asserting it

Separation has to be enforced by credentials and configuration, not by discipline. In a dbt project, environments are defined by the connection and target the project runs against, and a deployment environment is configured separately from the development environment.¹⁹ The evidence a reviewer should be able to see is:

• Distinct credentials per environment, with the production service account usable only by the orchestration platform and not by individual developers. If a developer’s personal credentials can write to the production schema, there is no separation regardless of what the diagram says.
• Distinct storage targets, meaning different databases or projects rather than different schema prefixes in the same database. Prefix separation fails the first time someone mistypes a target.
• Production deployment from a tagged release, not from a branch tip. This is what makes the production state identifiable. “Production runs the code at the tip of main” is not a version. “Production runs tag v14.2, commit abc1234” is.
• Access review evidence showing who currently holds production write access, reviewed on a defined cycle. Identity and access management is one of the areas the Annex 11 revision draft expands on.⁷

Production data in lower environments

This is the question that comes up in every one of these programs and it rarely gets a clean answer. Developers want realistic data because synthetic data hides the problems that matter. Quality wants to know why regulated data is sitting in an environment with no controls on it.

Both positions are reasonable. The resolution is to stop treating it as a single question. It splits into three, and each has a different answer.

Whichever combination is chosen, the decision needs to be written down with its rationale, because “we copy production into dev” discovered during an inspection without a documented basis is a finding, while the same practice with a documented risk assessment and access controls usually is not.

The promotion path and who approves a release

The promotion path is where change control and continuous integration have to be reconciled. Continuous integration in dbt typically builds only the modified models and their downstream dependents into a temporary schema when a pull request is opened, so the change is verified before it merges.²⁰ That is the mechanism. The governance sits around it.

Two details in that sequence do most of the work. Deploying from a tag rather than from a branch means the thing that was approved is exactly the thing that runs, which is the entire point of a release. And keeping the QMS change record even though the pull request already documents the change means the quality system retains a coherent index of changes across all systems, not just the ones that happen to live in a repository. The pull request is the technical record. The change record is the quality record. They reference each other and neither replaces the other.

Rewriting the Validation Deliverables

Quality organizations think in deliverables, and the fastest way to unblock a program is to show which traditional deliverable each piece of the code-based approach satisfies, and which ones still have to be written as documents. Some do.

The pattern in that table is consistent. The specifications and the test evidence move into the repository and get better in the process. The judgments stay as documents, because a judgment is a person taking responsibility and no pipeline does that.

What about the framework itself

One question comes up in every review and has a settled answer. The transformation framework is infrastructure or a standard software product depending on how it is deployed, and it is not custom application code. Under GAMP 5 categorization, the framework and the warehouse are supplier-provided components qualified through supplier assessment and infrastructure qualification, while the models an organization writes are the configured or custom layer that carries the qualification effort.¹⁴ Nobody is being asked to validate the query compiler. They are being asked to validate what their team wrote with it, which is a much smaller and more sensible task. Supplier assessment of the platform vendor still applies, and the Annex 11 revision draft puts more weight on supplier oversight than the current text does.⁷

Secure development practices belong in the same conversation

Because the transformation layer is now software, the practices that apply to software apply here: dependency management, secrets handling, review requirements, and build integrity. NIST’s Secure Software Development Framework provides a recognized structure for those practices that most IT organizations already reference for other systems.²⁶ Aligning the analytics repository with the same standard avoids inventing a parallel set of controls and makes the security review shorter.

A Practical Sequence for Getting There

Most organizations are not starting from nothing. There is usually an existing analytics platform, a data team already working in a repository, and a quality organization that has not yet been asked about it. The sequence that works starts with scope rather than with controls, because controls applied before scope get applied to everything and then abandoned.

This is not really about dbt

dbt is the example because it is what most teams are using, but nothing in the argument depends on it. The same reasoning applies to any framework where transformations are code in a repository: SQL-based frameworks, orchestrated processing jobs, statistical programs in clinical development, or a warehouse-native transformation service. What matters is that the logic is expressed as reviewable text, that changes go through a controlled path, that the tests are meaningful and can be shown to fail, that scope is asserted and enforced, and that the record outlives the tool. Every one of those is a property of the practice, not the product. An organization that gets those right can change frameworks without redoing its validation approach, which is itself a good argument for building the approach around the practice rather than around the vendor.

Conclusion

The collision between analytics engineering and validation culture is mostly a collision of formats, not of substance. A transformation layer under version control, with review before merge, tests that run on every build, and deployment from tagged releases, produces a stronger and more complete change record than the document-and-screenshot process it replaces, and that record maps cleanly onto ALCOA+ and onto the risk-based direction that FDA, EMA, and PIC/S have all been moving toward. The organizations that struggle are usually not struggling with a regulatory obstacle. They are struggling because nobody has translated what the pipeline already produces into the language the quality system uses.

The parts that are genuinely hard are narrow and worth naming plainly. Repository history is immutable by convention rather than by construction, so branch protection and signed commits become controls with owners rather than developer preferences. A repository is not a records archive, so releases have to be exported and retained on a defined period. And scope has to be asserted deliberately and enforced in the build, because a warehouse has no natural boundary and an unclassified model is a defect rather than a neutral state. Close those three and the rest of the approach stands on evidence that is better than what most validated reporting systems can produce today.

Sakara Digital works with pharma and biotech organizations building analytics platforms that have to satisfy both an engineering team and a quality organization. If you are moving your transformation layer into code and want an independent perspective on how to scope it, what to control, and what evidence to keep, we are happy to have that conversation.

For Further Reading