Beyond the SOP Index: Four Data Quality Pilots for Document Management

The Metadata Nobody Owns

Ask a quality director how many controlled documents the company has and you will usually get a number. Ask how many of those documents have a current, named owner who still works there, and the conversation changes. The number is knowable, the system will produce it in an afternoon, and almost nobody has looked.

This is the shape of the problem. Document management systems in life sciences are built around content control: version, approval, effective date, training assignment, withdrawal. That machinery works. What surrounds the content is a set of descriptive fields that tell you what a document is, who is responsible for it, what part of the business it belongs to, which product or site or supplier it applies to, and when it should next be looked at. Those fields are metadata, and in most companies they are the only part of the document system that no function owns end to end.

How the fields degrade

Metadata rarely fails all at once. It degrades through a set of ordinary events, none of which looks like a quality problem at the time.

• Migration. A company moves from one document system to another and maps the old fields to the new ones as best it can. Values that had no home in the new model get dropped into a free-text field. The migration validation checks that documents arrived, not that their descriptions still mean anything.
• Free text where a picklist belonged. A “document type” field that accepts typing will, over ten years, accumulate SOP, S.O.P., Standard Operating Procedure, Std Operating Proc, and Procedure as five separate types. Every report that groups by type is now wrong.
• Local convention. Two sites join the same system through an acquisition, each carrying its own naming pattern for batch records and its own idea of what a “process area” is. Neither is asked to change, because changing looks like a project.
• People leaving. The owner field is populated with a person, not a role. When that person leaves, nothing in the system notices. The document keeps its owner. The owner keeps their name on a hundred documents nobody will review.
• Placeholder values that became permanent. A field made mandatory mid-life gets backfilled with “TBD” or “N/A” for the existing population so the system will accept the records. The backfill was supposed to be temporary.

The symptoms show up somewhere other than the document system. Periodic review lists are wrong, so documents past their review date are missed while current ones are reviewed twice. Training assignments miss a group because their process area was spelled differently. An inspection request for “all procedures governing aseptic gowning at this site” takes two days and three people to assemble, and the assembly is done by memory rather than by query. A supplier audit turns up a quality agreement that nobody can match to the products the supplier actually makes.

Why this is a data quality problem

Treat the document register as a dataset and the diagnosis becomes routine. Each document is a record. Each field is an attribute with an expected domain of values. Completeness, validity against that domain, currency, internal consistency, and linkage to other records are all measurable in a query. The work is not new; it is the same profiling that a data team would run against a customer master or a materials master.

The research literature on metadata quality has been doing exactly this for two decades. Ochoa and Duval’s work on automatic evaluation of metadata quality in digital repositories defines completeness metrics that count populated fields per record and add a weighting factor so that a field of higher relevance carries more of the score, then tests those automated metrics against human review.¹⁴ That is the mechanism behind the scorecard later in this article: not every field matters equally, and a score that pretends they do will point remediation at the wrong place.

The FAIR guiding principles, published in Scientific Data in 2016, make the same point from the other direction. Findability depends on persistent unique identifiers and on data being described with rich metadata that is indexed in a searchable resource.¹³ FAIR was written for research data, not for a GMP document system, and it is not a regulatory expectation. It is a useful reminder that findability is a property you design and measure, not a property you hope for.

What Documentation Control Actually Requires

The regulations do not contain a chapter called metadata. They contain a set of requirements that cannot be met without it. Reading them that way is the useful move, because it converts a housekeeping project into a compliance argument that a quality council will fund.

The US requirements

21 CFR 211.100(a) requires written procedures for production and process control, and requires that those procedures be drafted, reviewed, and approved by the appropriate organizational units and reviewed and approved by the quality control unit.³ 211.22(d) requires that the responsibilities and procedures applicable to the quality control unit be in writing and followed.⁴ Neither obligation can be demonstrated at scale unless the system can tell you, for a given procedure, who owns it and which organizational unit approved it.

21 CFR 211.186 is the strongest metadata requirement in Part 211, and it is not usually read that way. Master production and control records must be prepared for each drug product, including each batch size, and must be prepared, dated, and signed by one person and independently checked, dated, and signed by a second. The preparation of those records must itself be described in a written procedure, and that procedure must be followed. The content requirements begin with the name and strength of the product and a description of the dosage form.² Product, strength, dosage form, and batch size are the exact fields a naming convention needs. The regulation put them there.

The European and PIC/S requirements

EU GMP Chapter 4 covers documentation, and the PIC/S Guide to Good Manufacturing Practice PE 009-17 Part I carries Chapter 4 with the same clause numbering.⁵ Chapter 4 is currently under revision alongside Annex 11 and the draft Annex 22, and that revision is analyzed in a separate article in this series. For present purposes the clauses that matter are the ones that have been in force for years.

• 4.1 requires that all types of document be defined and adhered to, with the requirement applying equally to all document media types. That is a document type taxonomy, stated as an obligation.
• 4.3 requires that documents containing instructions have unambiguous contents, be uniquely identifiable, and have a defined effective date. Unique identification and effective dating are metadata fields before they are anything else.
• 4.5 requires that documents within the quality management system be regularly reviewed and kept up to date. The PIC/S text adds that when a document has been revised, systems should be operated to prevent inadvertent use of superseded documents.⁵
• 4.10 requires that it be clearly defined which record relates to each manufacturing activity and where that record is located.
• 4.32 requires that an inventory of documents within the quality management system be maintained. That inventory is the dataset the four pilots below operate on.

For outsourced activities, EU GMP Chapter 7 requires a written contract that specifies the respective responsibilities and communication processes of the contract giver and contract acceptor, and requires that the contract clearly describe who undertakes each step of the outsourced activity, including knowledge management, technology transfer, supply chain, subcontracting, purchasing and testing of materials, and production and quality controls.⁶ A contract that describes all of that and is filed with a counterparty name and nothing else has satisfied the drafting requirement and defeated the retrieval one.

ICH Q7 and ICH Q10

ICH Q7 section 6 sets out the documentation system for active pharmaceutical ingredients. All documents related to manufacture should be prepared, reviewed, approved, and distributed according to written procedures, and the issuance, revision, superseding, and withdrawal of all documents should be controlled with maintenance of revision histories.⁸ Q7 6.41 requires master production instructions to carry the name of the intermediate or API being manufactured and an identifying document reference code where applicable. Q7 6.50 requires that a batch production record be checked before issuance to confirm it is the correct version and an accurate reproduction of the appropriate master production instruction, and that where the batch record comes from a separate part of the master document, that document reference the current master production instruction in use.⁸ Version control that depends on a person reading a title correctly is version control resting on metadata.

ICH Q10 supplies the management frame. Q10 treats knowledge management as an enabler of the pharmaceutical quality system, describing it as a systematic approach to acquiring, storing, and sharing information about products, manufacturing processes, and components across the product lifecycle.⁷ Q10 section 1.8 asks for a quality manual or equivalent documentation approach that describes the quality system, including identification of the quality system processes and their sequences, linkages, and interdependencies.⁷ A process taxonomy is implied in that sentence. Most companies wrote the quality manual and never connected its process list to the process area field in the document system, which is why the two disagree.

What inspection findings show

Public FDA warning letters do not cite metadata fields. They cite the consequences of not being able to produce, control, or trust documents, and the pattern is consistent.

In a March 2026 warning letter to a contract testing laboratory in Hyderabad, FDA described investigators finding torn analytical records, including chromatographic results, in garbage bags that the firm attempted to remove at the start of the inspection, and laboratory staff using unofficial personal diaries to record procedures, observations, results, method modifications, and deviation descriptions. FDA noted that the firm lacked procedures to control the use of those diaries.¹⁰ The finding is about uncontrolled documents: records that existed, mattered, and were outside the system that was supposed to know about them.

In an August 2026 warning letter to an API manufacturer in Tianjin, FDA cited failure to prepare master production and control records, describing batch records that lacked critical processing information, and noted that formal procedures had not been finalized because products were described as still in development.¹¹ In a June 2026 warning letter to a manufacturer in Navi Mumbai, FDA cited quality unit failures around oversight of contract testing laboratories, referenced its quality agreements guidance, and repeated its standing position that a manufacturer is responsible for the quality of its drugs regardless of agreements in place with contract facilities.¹²

Pilot One: SOP Metadata Standardization

Scope: one site or one function, two to four weeks, no change to the content of a single SOP.

The first pilot is the one to run first, because it produces the vocabulary the other three depend on. The goal is narrow: give the SOP population a defined set of document types, a controlled vocabulary for process area, and a completeness score that can be measured before and after.

The three artifacts

A document type taxonomy. Chapter 4.1 requires that all types of document be defined. Most companies have that definition somewhere in the document control SOP and a different, larger set of values in the system. The pilot reconciles the two. Keep the list short. Eight to fifteen types covers a GMP document population, and every type needs a one-line definition that says what belongs in it and, more usefully, what does not.

A controlled vocabulary for process area. This is the field that decides whether a search returns the right twelve documents or the wrong four hundred. Build it from the quality system process list in the quality manual, which ICH Q10 already asks you to maintain,⁷ rather than from the organization chart. Organization charts change every two years. Processes do not.

The right-hand column matters more than it looks. A controlled vocabulary succeeds or fails on the mapping from the values people actually typed to the values you intend to keep. Write that mapping down as a lookup table during the pilot, because you will need it again when the next system migration happens.

Clinical operations has a working example of this discipline that manufacturing teams rarely look at. The TMF Reference Model, a community effort now part of CDISC, organizes trial master file content into zones, sections, and artifacts with defined sub-artifacts, so that hundreds of sponsors, CROs, and technology vendors can describe the same document the same way.¹⁵ Nothing about that idea is specific to clinical trials. It is a shared taxonomy with definitions, and it works because the definitions are written down and the values are constrained.

A metadata completeness score. The third artifact is the measure. Pick six to nine fields, weight them, and score each document. The scorecard section below sets out the method.

Running the four weeks

Pilot Two: The Document Owner Audit

Scope: the full controlled document population at one site, two to three weeks.

The second pilot answers one question: does every controlled document have a current, named owner who is still employed and still in a role that makes sense for that document? The measure is the orphan rate, and it is the single most quotable number in this article, because everybody understands it and nobody has it.

Defining an orphan precisely

The definition has to be written before the extract runs, or the result will be argued rather than acted on. Four categories are enough.

The method

Extract the register with the owner field, join it against the active employee roster from HR, and classify. The join is the whole technique. It takes an analyst a day, and it converts a subjective worry into a count. Then work the list in this order: departed owners first because they are unambiguous, empty owners second, generic owners third, mismatches last because they need judgment.

Reassignment should be made against a role, with a named incumbent recorded alongside it. A document owned by “Manager, Sterile Manufacturing” with the current incumbent named survives the next departure; a document owned only by a person does not. Where the system has one field, use a naming pattern that carries both. Where it has two, populate both and validate the incumbent against the roster on a schedule.

The regulatory case for this pilot is short and strong. Chapter 4.5 requires documents within the quality management system to be regularly reviewed and kept up to date.⁵ Review is an activity a person performs. 21 CFR 211.100(a) requires procedures to be drafted, reviewed, and approved by the appropriate organizational units and reviewed and approved by the quality control unit,³ and 211.22(d) requires the quality unit’s responsibilities and procedures to be in writing and followed.⁴ A document with no owner has no route to either.

Pilot Three: Master Batch Record Naming Across Sites

Scope: one product family across all sites that make it, or all master batch records at two sites, three to four weeks.

The third pilot is the one with the clearest regulatory anchor and the highest political difficulty, because naming conventions are where sites defend their local practice hardest. It helps to start from the point that the fields in question are not a preference. 21 CFR 211.186 requires master production and control records for each drug product including each batch size, and requires the record to state the name and strength of the product and a description of the dosage form.² ICH Q7 6.41 asks for the name of the intermediate or API and an identifying document reference code.⁸ Chapter 4.3 asks that documents be uniquely identifiable with a defined effective date, and 4.17 requires the manufacturing formula to carry the product name with a reference code relating to its specification, plus the pharmaceutical form, strength, and batch size.⁵

Put those together and the required fields are already listed: site, product, dosage form, strength, batch size, document type, version. The pilot’s job is to make sure each of those lives in its own metadata field, and that the human-readable title is generated from those fields rather than typed independently.

A structured identifier

A workable pattern is a fixed-order identifier with a stable separator, plus a title built from the same values.

The result is an identifier such as ATH-MBR-P0417-TAB-050MG-250K, with the version carried by the system. The title reads as a sentence built from the same fields, so that a person scanning a list sees the same facts the query sees.

The third row is the reason this pilot pays for itself. Two of those records looked like the same document under three names, and one of them is a different batch size, which under 211.186 is a different master record entirely. Naming inconsistency does not just make search harder. It hides a distinction the regulation requires you to make.

Pilot Four: Quality Agreement Metadata

Scope: every active quality agreement in the company, two to three weeks. This population is small, usually between thirty and three hundred documents, which is why it makes such a good pilot.

Quality agreements are typically filed as PDFs with two pieces of metadata: the counterparty’s name and the date somebody uploaded it. What they govern is inside the document. That arrangement works until somebody asks a question that requires the set to be read as data: which agreements cover this product, which suppliers have no agreement, which agreements have not been reviewed since the last change of scope, which agreements name a legal entity we no longer buy from.

FDA’s guidance on contract manufacturing arrangements says a quality agreement describes the owner’s and the contract facility’s roles and manufacturing activities under CGMP, defines key roles and responsibilities, establishes expectations for communication with key contacts for both parties, and specifies which products or services the owner expects from the contract facility and who has final approval for various activities. It lists the sections most quality agreements contain: purpose and scope, definitions, resolution of disagreements, manufacturing activities, and the life cycle of and revisions to the agreement. It notes that quality agreements may be reviewed during inspections.⁹

The guidance also describes product-specific considerations that a comprehensive agreement may address, including product and component specifications, defined manufacturing operations including batch numbering processes, responsibilities for expiration and retest dating, storage, shipment and lot disposition, and responsibilities for process validation.⁹ Every one of those is a reason the agreement needs to be findable by product, not only by supplier.

EU GMP Chapter 7 adds the requirement that the contract describe clearly who undertakes each step of the outsourced activity.⁶ And the enforcement record shows why this matters: FDA’s June 2026 letter to a manufacturer in Navi Mumbai cited quality unit failures in the oversight of contract testing laboratories and restated that a manufacturer remains responsible for the quality of its drugs regardless of agreements in place with contract facilities.¹² An agreement you cannot locate against a supplier and a product is not oversight.

The metadata set to add

The Metadata Quality Scorecard

Four pilots produce four sets of numbers. Leadership will remember one. The scorecard’s job is to be that one, and to survive contact with a quarterly review a year from now when the person who ran the pilots has moved on.

Five dimensions

Keep the dimensions few and defined in a sentence each, so that a number can always be traced back to a rule.

The weights are a starting point, not a standard. What matters is that they are decided once, written down, and applied the same way each quarter, because the value of the score is entirely in the comparison. Ochoa and Duval’s finding that weighted completeness tracks human judgment better than a flat field count is the reason to weight at all.¹⁴ If everything counts the same, an optional note field can offset a missing owner, and the score stops meaning anything.

How to report it

Report three things and no more: the score now, the score before, and the single worst-performing field. The third item is what turns a metric into an action, because it tells the next person where to start. Report the tenth percentile alongside the mean, since a mean of 82 percent can hide two hundred documents scoring under 40 percent, and those two hundred are the ones that will be asked for.

An honest word about benchmarks

There is no published industry benchmark for document metadata completeness in pharmaceutical quality systems, and any consultant who offers you one has made it up. The comparison that matters is against yourself: the same query, the same rules, the same population, three months apart. That is a real measurement, and it is more persuasive to a board than a borrowed number nobody can source.

Holding the Gains After the Pilot Ends

Most metadata pilots work. Most metadata pilots also decay, and they decay for a reason that has nothing to do with whether the cleanup was done well. The cleanup fixed the stock of documents. Nothing changed about the flow. Six months of new documents arrive under the old habits and the score drifts back toward where it started.

Five things prevent that, and all five are small enough to complete in the week after the pilot closes.

What to do after the four pilots

The four pilots are deliberately shallow and deliberately visible. Once they are done and the numbers have been reported twice, the organization has three things it did not have before: a controlled vocabulary, a measure, and a demonstrated ability to change a metadata population without breaking anything. That combination is what a larger program needs in order to be approved, and it is the reason to run the pilots in this order rather than starting with a governance charter.

Natural next steps, each of which is easier because the pilots ran: extend the type taxonomy to records as well as instructions, so that Chapter 4.10’s requirement to define which record relates to each activity and where it is held becomes a query rather than an interview;⁵ connect the process area vocabulary to the training curriculum so that assignment follows the same taxonomy; and apply the same profiling to the specification and analytical method populations, where the linkage dimension usually scores worst.

One further piece of practice is worth borrowing from engineering. ISPE’s good practice guide on the management of engineering standards treats a standard as having a full lifecycle from chartering through periodic review to retirement, with the review process being what keeps content current and compliant.¹⁶ Controlled documents deserve the same explicit lifecycle, including the retirement step. A large fraction of the metadata problem in an old document system is documents that should have been retired years ago and instead kept collecting owners, review dates, and search results.

Conclusion

The reason document metadata stays broken is not that it is hard. It is that it belongs to nobody and shows up in nobody’s objectives, while the work to fix it looks like administration rather than quality. The four pilots here are designed to break that framing by making the problem measurable in a fortnight and by tying each measure to something a regulator already requires: documents defined by type and uniquely identifiable, reviewed and kept current, retrievable when asked for, master records that state product, strength, dosage form, and batch size, and agreements that say who does what for which product. None of those obligations can be met by a system that cannot describe its own contents.

What makes these quick wins rather than a program is the scope discipline. No SOP content changes. No system is replaced. No governance body is created. A defined population, a written rule, a measured before and after, and a control that stops the problem returning. Do that four times and a company has both a materially better document system and the internal credibility to ask for something larger, which is exactly the order in which these things should happen.

Sakara Digital works with pharma and biotech organizations on data quality in regulated environments, including the unglamorous parts that decide whether an inspection week is calm or expensive. If you are considering where to start on document and record metadata, and want an independent view of which pilot would produce the clearest result in your systems, we are happy to have that conversation.

For Further Reading