RAG Architecture for Regulatory Content: Design Choices That Survive an Audit

Why RAG Demos Well and Audits Badly

The appeal of retrieval augmented generation is easy to understand. You take a question, convert it into a numeric representation, search a vector index for passages that are close to it in meaning, hand those passages to a language model along with the question, and ask the model to answer using only what it was given. The output looks like a knowledgeable colleague answering from the file cabinet rather than from memory. Compared with a general purpose chatbot, the improvement in accuracy on company-specific questions is large and immediate.

That first demo takes a competent engineer a few days. It is the reason RAG spread through pharma IT faster than almost any other pattern in the last three years. It is also the reason so many of these systems are now sitting in a gray zone, used daily by quality and regulatory staff, never formally validated, and unable to answer basic questions about their own behavior.

The research literature has been unusually blunt about this gap. An experience report drawing on three separate production RAG builds identified seven distinct failure points and concluded that validation of a RAG system is only feasible during operation, and that reliability evolves through use rather than being designed in at the start.¹ That is an uncomfortable finding for a regulated environment, where the expectation runs the other way: you establish that a system is fit for its intended use before you rely on it, and you keep evidence.

None of this argues against the pattern. It argues for treating the pattern as a system that has to be designed for auditability from the beginning, in the same way you would design a laboratory data system for data integrity rather than adding audit trails later. The five sections that follow are the choices that actually determine whether that is possible.

What “audit ready” means in this context

We use the phrase narrowly. An audit ready RAG system is one where you can answer four questions about any specific answer the system produced, at any time, without engineering effort:

• What content produced this answer? Not the document title. The document, the version, the effective date, and the specific passage.
• Was that content current at the time the answer was given? And if it has since been superseded, can you identify every answer that was based on the old version?
• Was the person who asked entitled to see that content? Enforced by the system, not by policy.
• How do you know the system works? A defined test set, measured performance, a threshold, and a record of what happened when performance moved.

Most implementations can answer the first question partially and none of the other three. The rest of this article is about closing that gap.

Design Choice 1: Document Versioning and Index Currency

This is the one that matters most and gets the least attention, so we are putting it first.

Here is the failure in plain terms. Your document control system holds SOP-QA-014 at version 6, effective March. Someone runs an ingestion job that pulls the effective version of every controlled SOP, splits each one into passages, converts those passages into vectors, and loads them into an index. The system works well. In July, SOP-QA-014 goes to version 7 through change control, with a meaningful change to a hold time limit. Nothing in the retrieval pipeline notices. The index still holds version 6 passages. The system keeps answering the hold time question with the superseded limit, confidently, with a citation to SOP-QA-014, which is a real and current document number. Every part of that answer looks right except the part that matters.

What has to exist in the architecture

Three things, and they are separable.

Version-aware chunks. Every passage in the index carries the document identifier, the version, the effective date, the status (effective, superseded, retired, draft), and the section or clause it came from. This metadata travels with the passage into the model context and out into the answer. It is not stored in a side table that a developer can join to later. It is attached to the unit of retrieval, because the unit of retrieval is what ends up in front of the user. A system that stores only a document title and a file path has already lost the ability to answer the auditor’s first question.

Index rebuild triggered by document control, not by a schedule. This is the design decision that separates the serious implementations from the rest. A nightly or weekly reindex job is a schedule. It means that between the moment a new version becomes effective and the moment the job next runs, the system is knowingly answering from superseded content. In a regulated setting that window is a finding waiting to happen, and the length of the window is arbitrary rather than justified. The alternative is an event-driven pipeline: the document management system emits an event when a document changes state, and that event drives the reindex for that document. The state change is the trigger. The schedule becomes a safety net that catches missed events, not the primary mechanism.

This has a practical consequence that people underestimate. It means the RAG index becomes a downstream consumer of document control, with all that implies: an interface that has to be specified, tested, and monitored; a defined behavior when the interface fails; and a reconciliation process that periodically compares what document control says is effective against what the index actually holds. That reconciliation report is one of the most useful artifacts you can hand an auditor, and almost nobody builds it.

The ability to state which version produced a given answer. Not “the system uses current SOPs”. The specific version, recorded at answer time, in a log you can query. The reason this has to be recorded rather than reconstructed is that the index changes. If you try to reconstruct which version was retrieved by asking the index today, you will get today’s answer, not the answer that was given in March. The retrieval event has to be logged with the version identifiers of the passages that were returned, alongside the question, the answer, the user, and the timestamp.

The change impact question

Once versioning is in place, a second capability becomes possible and is worth building deliberately. When a document version changes, you can identify the questions that were previously answered from the superseded passages and flag them. In practice this supports two things. It lets quality assess whether anyone acted on guidance that has since changed. And it gives you a targeted regression test set: the questions most affected by this change are exactly the questions to rerun after reindexing.

Regulatory documents are not static, and the research community has noticed the same thing. Work on regulatory question answering has specifically called out the length, complexity, and frequent updates of regulatory text as the defining difficulty of the domain, and built dedicated datasets to study it.¹¹ The currency problem is not a pharma quirk. It is the central problem of putting a model over any regulated corpus.

Design Choice 2: Citation Fidelity, Not Citation Presence

Every RAG product screenshot shows citations. Small numbered markers, a source panel, a link back to the document. The presence of a citation is what makes the output feel trustworthy, which is precisely the problem, because a citation is easy to produce and hard to verify.

There are two distinct failure modes here and they are usually confused with each other.

Citation correctness asks whether the cited passage actually supports the claim. A system can generate a fluent, reasonable answer and attach a citation to a passage that is topically related but does not state what the answer says. The user checks that a citation exists, sees a real document number, and moves on.

Citation faithfulness asks something harder: whether the model actually used the cited passage to produce the answer, or answered from its own training and then attached a citation that happened to fit. Researchers call the second behavior post-rationalization. In a study that separated these two properties for the first time, the authors found that up to 57 percent of citations in attributed answers lacked faithfulness.² The citation can be correct and unfaithful at the same time. The answer came from somewhere else, and the reference was attached afterward.

For a regulated use, the practical implication is direct. Telling the model to cite its sources is not a control. A prompt instruction produces citations. It does not produce evidence that the citations are load bearing. If the only thing standing between your quality organization and a wrong answer is an instruction in a prompt template, you do not have a control, you have a preference.

What an actual verification step looks like

Verifying that a retrieved passage genuinely supports a generated claim is a real engineering problem with real approaches, none of which are free.

Several public frameworks formalize the measurement side of this. RAGAS defines faithfulness and answer relevance alongside separate retrieval measures and evaluates them without needing hand-written reference answers.⁴ ARES takes a similar path with trained lightweight judges and confidence intervals.⁵ Earlier work established automatic metrics for citation quality in generated text and showed that models frequently produce citations that do not fully support their statements.³ There is also a purpose-built corpus of annotated hallucinations in retrieval augmented responses, which is useful if you want to train or calibrate a checker rather than assemble one from scratch.¹²

Design Choice 3: Access Control Enforced During Retrieval

A regulated document set is never uniformly readable. It spans products at different stages, sites with different scopes, partner content under confidentiality terms, unblinded clinical material, and human resources or legal documents that got swept into the same repository. The permission structure that governs it in the document management system is real and enforced. The permission structure in a RAG pipeline is frequently an afterthought.

The failure looks like this. Retrieval runs against the whole index, unfiltered, because filtering is harder and hurts relevance. Passages come back. They are placed in the model’s context. Then, at the end, something checks whether the user is allowed to see the sources and hides the ones they are not entitled to.

By that point the content has already been used. The model read the restricted passage, reasoned over it, and the answer it produced reflects it. Hiding the citation hides the pointer, not the information. A user with no rights to a partner’s process description can receive an answer that is materially shaped by it, correctly, and never know a boundary existed. This is not a theoretical concern: it is the specific reason enterprise RAG security work has argued that authorization has to be evaluated before content reaches the model rather than after.⁹

What this requires structurally

Filtering at retrieval time is straightforward in principle and fiddly in practice. Each passage carries the access attributes of its source document. The query carries the identity and attributes of the requesting user. The retrieval engine applies the filter as part of the search rather than after it. The standard vocabulary for this is attribute based access control, where decisions are made by evaluating attributes of the subject, the object, and the context against a policy, rather than by assigning users to fixed roles.¹⁰ That model fits regulated document sets well, because entitlement in pharma is rarely a simple role. It is a combination of study, site, product, partner agreement, blinding status, and often a time boundary.

Three implementation details decide whether this holds up.

Permissions have to be current, not copied. If access attributes are copied into the index at ingestion time and never refreshed, then a permission revoked in the document management system stays effective in the retrieval layer until the next reindex. That is the same currency problem as document versions, applied to entitlement, and it fails the same way. Either the filter resolves permissions live against the source of truth, or permission changes trigger an update the same way version changes do.

The filter has to be enforced server side. A permission filter constructed in application code that a client can influence is not a control. This sounds obvious and is violated constantly, usually because the retrieval service was built as an internal component and later exposed through an interface that accepts a filter parameter.

Denied results should be counted, not silenced. When a query would have returned content the user cannot see, that fact is worth logging and worth surfacing in aggregate. Not the content, and not per query in a way that lets someone probe the boundary, but as a monitored signal. A sharp rise in filtered results for a particular user or team is a useful thing to notice.

That last row deserves a note. Separate indexes for genuinely hard boundaries, for example unblinded study content or a single partner’s material under a confidentiality agreement, are often the right answer even though they duplicate infrastructure. The boundary becomes physical rather than logical, and physical boundaries are much easier to demonstrate to an auditor than policy logic. The trade-off is that you now have several indexes to keep current, which multiplies the versioning work described in the previous section.

Design Choice 4: Chunking Regulatory Text Without Breaking It

Chunking is the step where a document is split into retrievable pieces. It is treated as a tuning parameter, something you set to 512 tokens with 50 tokens of overlap and revisit if results are poor. For regulatory text it is a correctness decision, because the way procedural and regulatory language is written makes certain splits actively dangerous.

The four splits that produce confidently wrong answers

A requirement separated from its exception. Regulatory and procedural text is full of constructions like “Samples shall be tested within 24 hours of collection” followed two sentences later by “except for stability samples held under the conditions in Section 7.3, which shall be tested within 72 hours.” Split between those two statements and you have created a passage that states an unconditional rule the source document never stated. Retrieval returns it. The model answers with it. Every downstream check passes, because the passage genuinely says what the answer says. The document does not.

A table separated from its header. Tables carry meaning in two dimensions and most chunkers flatten them into one. Split a table across chunk boundaries and the rows in the second chunk lose their column names. A row that reads “Grade 2 | 15 | 30” is meaningless without the header that says what those columns are, and worse than meaningless if the model guesses. Regulatory content is dense with tables: acceptance criteria, sampling plans, stability schedules, dose modification rules.

A clause separated from its scope statement. Procedures often establish scope once at the top and then write the body in the imperative. A passage from the middle of a site-specific SOP reads as a universal requirement because the sentence limiting it to one site or one product family sits on page one and is nowhere near the chunk that was retrieved.

A definition separated from its use. When a document defines a term narrowly and then uses it throughout, a chunk containing the usage carries the ordinary meaning of the word rather than the defined one. “Critical” and “significant” do a lot of load-bearing work in quality documents, and their local definitions rarely travel with the passage.

What to do instead

Split on document structure, not character count. Use the section and clause hierarchy the document already has. Regulatory and procedural documents are heavily numbered for exactly this reason. A chunk that maps to a numbered clause is a chunk whose boundaries were chosen by the author.

Attach the ancestry to every chunk. Each passage carries the document title, the section path above it, and the scope statement. This is sometimes called contextual retrieval: prepending explanatory context to each chunk before it is converted into a vector, so that the chunk is interpretable on its own. Published measurements of this approach reported a 35 percent reduction in the top-20 retrieval failure rate from contextual embeddings alone, 49 percent when combined with a keyword search method, and 67 percent when a reranking step was added.⁸ The numbers come from a general corpus rather than a regulatory one, but the mechanism is the same and the mechanism is what transfers.

Handle tables as tables. Keep a table whole where possible. Where a table is too large, repeat the header row in every fragment and label the fragment. A chunker that treats a table the same as a paragraph has a known defect, not a tuning opportunity.

Carry exceptions with their requirements. Where structural splitting would separate a rule from a proviso, extend the chunk. Overlapping chunks are cheap. A missing exception is not.

Retrieve the neighborhood, not just the hit. When a passage matches, pull the adjacent passages and the parent section summary along with it. This uses up context length and buys correctness. Be aware that context length is not free in a different way: language models attend unevenly across long inputs, with performance highest when relevant information sits near the beginning or end of the context and degrading noticeably when it sits in the middle.⁷ Stuffing thirty passages into the context to be safe can make the answer worse, not better.

Design Choice 5: Evaluation and Monitoring You Can Show Someone

Ask most teams how they know their RAG system works and you will hear that the users like it and there have not been complaints. That is not an answer a quality organization can use, and it is not an answer the underlying technology makes hard to improve on.

The held-out question set

The foundation is a set of questions with known correct answers, written by the people who own the source content, and held separate from the people building the system. Two hundred questions is a reasonable target for a departmental system. The properties that matter:

• Each question has a documented correct answer and a documented correct source, identified by document, version, and section. This lets you measure retrieval and generation separately.
• The set includes questions the system should refuse. Questions whose answers are genuinely not in the corpus. If every question in your test set is answerable, you have no measurement of whether the system knows its own limits.
• The set includes the hard cases deliberately: requirements with exceptions, near-duplicate procedures across sites, terms that mean different things in different documents, and questions that require combining two documents.
• The set is versioned and maintained. When source documents change, the expected answers change. A test set that silently goes stale is worse than no test set, because it reports success.

Measure retrieval and generation separately

This is the part teams most often get wrong, and it has a practical consequence: if you only measure end-to-end answer quality, you cannot tell whether a regression came from the retriever or the generator, and you will spend weeks tuning the wrong component.

Separating retrieval quality from answer quality is standard practice in the evaluation literature, and the public frameworks are built around exactly this split.⁴⁵ Retrieval performance itself has a long measurement tradition, and benchmark suites exist specifically to test how retrieval models generalize to domains they were not trained on, which is directly relevant when you point a general purpose embedding model at pharmaceutical quality documents.¹³ The same applies to the choice of embedding model itself, where public comparisons across many tasks are available and worth consulting before defaulting to whatever the platform ships with.¹⁴

Detecting degradation when the corpus moves

A RAG system does not degrade the way a piece of software degrades. The code does not change. The corpus changes underneath it, and performance moves without anyone deploying anything. Four signals are worth monitoring continuously.

Where This Is Genuinely Worth Doing in Pharma

All of the above is real work. It is worth being specific about where the return justifies it, because the pattern is being applied indiscriminately and some applications do not need this much machinery while others need more.

SOP and policy question answering

This is the highest-volume use and the one with the clearest value. A manufacturing supervisor, a QA reviewer, or a new analyst has a procedural question and the answer exists in a controlled document that takes fifteen minutes to find. Multiply by a few thousand questions a month across a site.

It is also the use with the sharpest failure consequences, because the answers drive actions on product. This is where every one of the five design choices earns its place, and where the versioning requirement is non-negotiable. If you build only one audit-ready RAG system, build this one, and accept that it is a validated system with the documentation that implies.

Regulatory intelligence across authority guidance

Regulatory affairs teams track guidance across FDA, EMA, MHRA, ICH, and national authorities, in several languages, updated continuously. The corpus is external, large, and genuinely hard to keep current manually. RAG fits well here.

The design emphasis shifts. Access control matters less, because the source content is public. Versioning matters differently: you are tracking draft versus final, comment periods, transition dates, and superseding documents, and the currency question becomes “which version of this guidance was in force on the date in question.” Citation fidelity matters enormously, because the output feeds regulatory strategy decisions and a mis-attributed requirement can send a submission in the wrong direction. Published work in this area has built dedicated datasets and metrics precisely because generic evaluation does not capture whether an answer covers all applicable obligations without contradicting itself.¹¹

Submission content reuse

Protocols, clinical study reports, statistical analysis plans, and module summaries share large amounts of methodology and safety language. Finding the right precedent text across a submission archive is slow, and inconsistency between documents that should agree is a real review risk.

Here the retrieval requirement is different again: you are usually looking for a passage to reuse rather than an answer to a question, which means recall and provenance matter more than generation quality. The system should be very good at finding candidate text and identifying exactly which document and version it came from, and comparatively conservative about rewriting it. Applied research on extracting structured product information from regulatory source documents using retrieval augmented approaches has shown the pattern works for this kind of targeted extraction, while also being clear that it is a preliminary result rather than a solved problem.¹⁷

Deviation and CAPA precedent search

When an investigator opens a deviation, the most useful question is whether anything similar has happened before, what the root cause was, and whether the corrective action worked. That information exists in the quality system and is nearly unsearchable, because deviation records are written in free text by many authors with inconsistent terminology.

This is the use where RAG adds the most and the risk profile is most manageable, because the output supports human investigation rather than replacing a decision. The design emphasis is on recall and on access control, since deviation records often carry product and site restrictions. Versioning is less central because quality records are point-in-time rather than versioned. The realistic goal is a shortlist of relevant prior events with links, not a generated root cause.

What the applied evidence shows so far

Peer-reviewed evaluation of RAG on pharmaceutical regulatory content is starting to appear and is worth reading carefully rather than quoting selectively. One 2026 study evaluated RAG systems against FDA guidance for drug information and clinical trial protocols. On protocol summarization it reported 100 percent answer relevancy and 95 percent faithfulness. On evaluating a statistical analysis plan against statistical guidance it reported 85.7 percent accuracy and 100 percent faithfulness, with a lower domain-specific score that the authors attributed to omissions of minor details in the statistical methodology.¹⁶

Two things are worth drawing from that. The first is that the technology performs respectably on well-scoped regulatory tasks. The second is the shape of the residual error: omissions rather than fabrications. For a system supporting a regulatory reviewer, a quiet omission is the harder failure to catch, because there is nothing wrong on the page. That is an argument for designing the human review step around completeness rather than around checking whether what is written is true.

The Questions an Auditor Will Ask

This is the section to take into a design review. These are the questions we would expect an inspector or an internal auditor to ask about a RAG system used in a GxP context, and the artifact that answers each one. If you cannot produce the artifact, the design is not finished.

Conclusion

Retrieval augmented generation is a good pattern and it is going to be widespread in pharma and biotech regardless of what anyone writes about it. The question is not whether to use it but whether the version you build can answer for itself. The five choices in this article are the ones that decide that, and they are all architectural. Versioning, citation verification, access control placement, chunking, and evaluation design are decided in the first two weeks of a build and are expensive to retrofit in month nine, which is exactly when someone asks the first hard question about it.

Our consistent observation is that the gap between a demo and a defensible system is not model quality. It is provenance and evidence. The teams that get this right treat the retrieval index as a controlled derivative of document control, treat the prompt template and the embedding model as controlled configuration, and build the held-out question set before they build the interface. That sequence feels slower for about a month and then stops being slower, because they are not rebuilding.

Sakara Digital works with pharma and biotech organizations designing this kind of system for regulated content. If you are standing up a RAG capability over SOPs, guidance, or quality records and want an independent read on where the architecture will be challenged, we are happy to have that conversation.

For Further Reading