Why the Release Package Needs Its Own Scorecard

Every batch of medicine ends the same way. Someone with the authority to release it looks at the evidence and decides. In the European Union that person is a Qualified Person, and the expectations are set out in EudraLex Volume 4, Annex 16, the European Commission’s guidance on QP certification and batch release, dated October 12, 2015 and in operation since April 15, 2016.1 In the United States the decision belongs to the quality control unit, which 21 CFR 211.22 gives the responsibility and authority to approve or reject drug products and to review production records.2 The titles differ. The task is the same: confirm, from records, that this batch was made and tested the way it was supposed to be, and that nothing open undermines that conclusion.

The quality of that decision depends on the quality of the data in front of the reviewer. That is the subject of this article. Not the quality of the process, and not the quality of any single system, but the completeness, accuracy, and linkage of the records that make up one batch’s release package at the moment they are handed over.

What the Reviewer Is Being Asked to Confirm

Annex 16 is unusually specific. Section 1.7 lists 21 points, numbered 1.7.1 to 1.7.21, that the QP is responsible for ensuring are secured before certification. They range from the broad (all manufacturing and testing was done under GMP) to the precise (all records are complete and endorsed by appropriate personnel; all investigations pertaining to the batch, including out-of-specification and out-of-trend investigations, have been completed to a sufficient level to support certification).1 The same section says these tasks may be delegated to trained personnel or third parties, and that the QP will rely on the pharmaceutical quality system but should have “on-going assurance that this reliance is well founded.”1

That phrase is the reason a release package scorecard exists. A QP cannot personally re-verify every entry in every record for every batch. The QP relies on systems and people upstream. A scorecard is one way to make that reliance visible and measurable, so that “assurance” is backed by data rather than by the absence of complaints.

The US regulation reaches the same point from a different direction. 21 CFR 211.192 requires that all production and control records, including packaging and labeling records, be reviewed and approved by the quality control unit “before a batch is released or distributed,” and that any unexplained discrepancy or failure to meet specifications be thoroughly investigated, with the investigation extended to other batches that may be affected.3 21 CFR 211.165 adds that each batch must have an appropriate laboratory determination of conformance to final specifications, including the identity and strength of each active ingredient, before release.4

Where Existing Scorecards Stop

Sakara Digital has written before about two related scorecards. One scores the data in a regulatory submission across six dimensions before filing. Another tracks manufacturing data quality through plant-level KPIs such as deviation rate reliability and MES-to-LIMS reconciliation. Both are useful, and neither answers the question a release reviewer asks every day. The submission scorecard works at the level of a dossier. The manufacturing scorecard works at the level of a site and a month. The release package is a third unit: one batch, all of its linked records, assembled at one point in time for one decision.

We have also covered the design of individual electronic batch record fields, such as timestamps, material lot entries, and calculations, that later trigger investigations. Field design matters, and it is upstream of this topic. A well-designed batch record can still arrive at the reviewer with a deviation that has no approved impact assessment, a laboratory result tested against a superseded specification, or a cleaning record dated after the equipment was used. Those are package problems. They come from the way records in different systems are linked, not from the way any one field was built.

What Enforcement Shows About Package Failures

FDA warning letters rarely use the word “package,” but many describe exactly this failure. In a February 9, 2026 warning letter to VeganicSKN Limited in Rocklea, Queensland, FDA described an out-of-specification (OOS) result reported by a contract laboratory on August 28, 2024, a Phase II investigation not started until December 4, 2024, and a quality unit that released the batch while the investigation was still open, after a retest gave passing results. The investigation closed on July 11, 2025 without a root cause. FDA redacted the release date in the published letter.5 The data needed to stop that release existed. The open investigation was simply not treated as a condition of the package.

In an August 22, 2025 letter to Wisconsin Pharmacal Company, an over-the-counter drug manufacturer, FDA described a batch released for distribution on a passing retest after Staphylococcus aureus was found, without a thorough investigation of the root cause, and a partial batch rejection in which about half of the rejected product was distributed anyway.6 Among the corrective actions FDA asked for was “A complete and final review of each batch and its related information before the QU disposition decision.”6 That sentence is close to a definition of what a release package scorecard is meant to support.

21Points in Annex 16 section 1.7 that a QP must ensure are secured before certifying a batch1
10–40Days that typical paper-based batch release can take, as described in BioPharm International in 20197
95%Share of batch records that should be right first time to support a one-day review, per a 2012 BioPharm International article8

Defining the Package and Its Six Data Domains

A scorecard only works if everyone agrees on what is being scored. The first job is to write down what a release package contains, where each part comes from, and which requirement it answers. We use six data domains. They are not the only possible split, but they line up with the systems most sites run and with the way reviewers think.

Data DomainWhat the Reviewer Needs to SeeTypical Source SystemMain Requirement Anchors
1. Batch record completenessEvery step executed, signed, and checked; yields and reconciliations within limits or explainedMES or paper batch record21 CFR 211.188, 211.192; Annex 16 point 1.7.11
2. Deviations and their closureEvery deviation raised during the batch linked, assessed for impact, and concluded to a level that supports releaseeQMS or deviation log21 CFR 211.192; Annex 16 points 1.7.16, 1.7.17 and section 3
3. Test results and specificationsAll required tests reported against the correct, approved specification, with any OOS or OOT result investigatedLIMS, chromatography data system21 CFR 211.165, 211.194; Annex 16 point 1.7.13
4. Environmental monitoringMonitoring results and trends for the classified areas and sessions used by the batch, with excursions investigatedEM module of LIMS, or a separate EM systemEU GMP Annex 1 paragraphs 3.2, 9.3 and 10.10
5. Supplier CoAs and material statusEvery material lot used was released for use, with identity testing and a supported basis for relying on the supplier’s certificateERP, LIMS, supplier quality records21 CFR 211.84; Annex 16 points 1.7.2 and 1.7.6
6. Equipment statusEquipment used was qualified, calibrated, cleaned, and fit for use at the time of useCMMS, calibration system, equipment logbooks21 CFR 211.67, 211.182; Annex 15 section 4; Annex 16 point 1.7.12

Annex 16 also includes points that do not belong to one domain: changes affecting the batch have been evaluated (1.7.15), ongoing stability data still supports certification (1.7.14), open complaints, investigations, or recalls do not negate the conditions for certification (1.7.17), and the required technical agreements are in place (1.7.18).1 We treat these as cross-cutting checks attached to the package rather than as a seventh domain, because they usually live outside any batch-specific record and are easiest to answer once per product rather than once per batch.

Drawing the Package Boundary

The boundary question sounds trivial and causes most of the arguments. Three rules help.

  • The package is defined by the batch, not by the system. If a deviation was raised against an upstream intermediate or a shared utility and could affect this batch, it belongs in this batch’s package, even if the deviation record lists a different batch number. The investigation scope requirement in 211.192 makes this a regulatory point as well as a practical one.3
  • The package has a time window. Environmental monitoring, equipment status, and calibration checks are only meaningful for the dates and sessions the batch used. Write the window down: from the first dispensing step to the last packaging step, plus any hold times.
  • The package includes the links, not just the records. A deviation number written in a batch record comment is not the same as a deviation linked in the eQMS to that batch. The scorecard should check for the link, because the link is what lets the reviewer, and later an inspector, follow the evidence.

Who Assembles, Who Scores, Who Decides

Three roles keep the scorecard from turning into a second review. The assembler, usually a production or quality operations coordinator, pulls the records together and resolves gaps with the record owners. The pre-checker, a trained person who is not the final reviewer, runs the scorecard checks and records results. The reviewer, whether a QA disposition reviewer or a QP, makes the release decision. Annex 16 expressly allows the supporting tasks to be delegated.1 What cannot be delegated is the decision itself, and the scorecard must never be presented as if it were the decision.

A practical test for the boundary. Ask your reviewers what they most often have to chase after a package arrives. The answers usually name four or five specific links: deviation impact assessments, laboratory investigation status, EM trend sign-off, and calibration dates. Those are the checks to build first.

Checks for Batch Record Completeness and Deviations

The checks in this section and the next three are written as yes-or-no questions that a trained pre-checker can answer from records. Each is marked with a proposed type. Sites should adjust the type to their own risk assessment, but the starting point should be conservative.

Batch Record Completeness

21 CFR 211.188 lists what batch production and control records must document, including dates, identification of major equipment, the specific batch of each component, in-process and laboratory control results, actual yield and percentage of theoretical yield, complete labeling control records, the people performing and checking each significant step, and any investigation made under 211.192.9 Annex 16 point 1.7.11 says all records are complete and endorsed by appropriate personnel, and all required in-process controls and checks have been made.1 The package-level checks follow from those lists.

  • Every step executed and signed, with no blank required fields (blocker). This includes second-person verifications and the identification of the person checking any step done by automated equipment.
  • Corrections follow good documentation practice, with a reason recorded (defect). A correction without a reason does not mean the batch is bad, but it will be questioned, and it is easier to fix before review.
  • Yield and reconciliation within the limits in the master record, or linked to a deviation (blocker). 211.192 names a yield outside the established percentages as an unexplained discrepancy that must be investigated.3
  • Label reconciliation complete and within limits (defect, or blocker for sites with a history of labeling events).
  • All in-process control results present and within limits (blocker).
  • The executed record matches the current approved master record version (blocker). A batch made on a superseded master is a change control question, not a documentation slip.

For sites running an electronic batch record, many of these checks are already enforced by the system at the point of entry. That is good, and it changes the type of check rather than removing it. The pre-checker no longer looks for blank fields. Instead, the pre-checker confirms that the exceptions the system raised were all dispositioned. Our earlier article on review by exception for batch records covers what must be in place before a site can rely on system-generated exceptions in this way.

Deviations and Their Closure

Deviations cause more release delays than any other domain, and they are where the gap between “closed” and “ready” is widest. The question for release is not whether every deviation is closed. Many deviations have CAPA actions that will run for months, and that is normal. The question is whether each deviation’s impact on this batch has been assessed and concluded to a level that supports the decision. Annex 16 point 1.7.16 uses that language: investigations completed “to a sufficient level to support certification.”1

Annex 16 section 3 adds detail for unexpected deviations from the marketing authorization or GMP. Provided registered specifications are met, a QP may consider certifying a batch after such a deviation, but the deviation should be thoroughly investigated, the root cause corrected, and the impact assessed through a quality risk management process that includes an evaluation of the potential impact on quality, safety, or efficacy and a conclusion that the impact is negligible.1

  • Every deviation raised during the batch window, or against shared materials, equipment, or utilities used by the batch, is linked to the package (blocker). The check runs from the deviation system toward the batch, not only from the batch record outward, because the batch record only lists the deviations someone remembered to note.
  • Each linked deviation has an approved impact assessment with a stated conclusion for this batch (blocker).
  • The scope of each investigation is stated, with the reason other batches were included or excluded (defect). 211.192 requires that investigations extend to other batches that may be associated with the failure.3
  • Any deviation classified as major or critical has QA approval of its classification (blocker).
  • Open CAPA actions linked to the deviation are listed, with a statement that they do not affect this batch’s disposition (defect).

The open investigation trap. The most dangerous state in a release package is an investigation that is open but no longer feels urgent, because a retest passed. In the VeganicSKN letter, the batch was released during an ongoing OOS investigation after a passing retest, and FDA called the response inadequate in part because it gave no justification for leaving the investigation incomplete for nearly 11 months.5 A scorecard should treat any open laboratory or manufacturing investigation linked to the batch as a blocker unless an approved, documented assessment states why the open part of the investigation cannot affect the batch.

Checks for Test Results, Specifications, and Environmental Monitoring

Test Results Against the Right Specification

The question the reviewer answers here sounds simple: did the batch meet its specification? The data problems are rarely in the result itself. They are in the context around the result. Was it tested against the specification version that matches the approved application? Were all required tests reported? Was the method version the validated one? Were the laboratory records reviewed by a second person, as 21 CFR 211.194(a)(8) requires?10

  • Every test on the approved specification has a reported result for this batch (blocker). Missing tests are more common than people expect when a specification has been revised and the LIMS product template was not updated in step.
  • The specification version used for testing matches the version in the approved marketing application, or the approved change (blocker). Annex 16 point 1.7.13 ties finished product test data to the finished product specification in the marketing authorization.1
  • Units, rounding, and reporting format match the specification (defect). A result reported to more decimal places than the specification allows can change a pass to a fail when it is rounded correctly.
  • Any OOS or out-of-trend result is linked to an investigation that has reached a conclusion supporting release (blocker).
  • Any retest or resample is covered by an investigation that justifies it (blocker).
  • Second-person review of laboratory records is complete (defect).
  • Results from contract laboratories have been received with their supporting data, not only a summary certificate (defect).

The retest check deserves emphasis. FDA’s 2022 guidance on investigating OOS results describes firms that test repeatedly until they get a passing result and then disregard the OOS results without scientific justification. The guidance states plainly: “This practice of ‘testing into compliance’ is unscientific and objectionable under CGMP.”11 The same guidance says that when a contract laboratory finds no clear laboratory error, it should pass its data to the manufacturer’s quality unit, which should then start the full-scale Phase 2 investigation.11 A release package that shows a passing retest with no linked investigation, or with an investigation still at the laboratory, should never reach the reviewer looking complete.

Environmental Monitoring for the Batch Window

For sterile products, the 2022 revision of EU GMP Annex 1, in operation since August 25, 2023 (with one paragraph deferred to 2024), is direct. Paragraph 10.10 says: “Environmental monitoring data and trend data generated for classified areas should be reviewed as part of product batch certification/release.”12 Paragraph 3.2 says that non-conformities such as sterility test failures, environmental monitoring excursions, or deviations from procedures should be adequately investigated before certification or release of the batch.12 Paragraph 9.3 says that information from environmental and process monitoring systems should be used for routine batch certification and release.12

Two features of the EM domain make it different from the others. First, the data is not batch-specific. A single EM session may cover several batches, and a single batch may span several sessions. The package needs an explicit mapping from the batch’s processing window to the sessions, rooms, and sample points that apply. Second, the data matures late. Microbiological results take days to incubate. Annex 1 recognizes this for short shelf-life products, saying that where EM data for the time of manufacture is not available, compliance should include a review of the most recent available data.12

  • The batch window is mapped to the EM sessions, rooms, and sample points that apply (blocker for sterile products).
  • All scheduled viable and non-viable results for those sessions are present, including personnel monitoring (blocker).
  • Every action limit excursion in the window is linked to an investigation with an impact conclusion for this batch (blocker).
  • Alert limit results and adverse trends in the window have been reviewed and the review is signed (defect).
  • Where results are pending at the time of assembly, the pending items are listed with expected dates (observation, so the reviewer can see what is outstanding).

The reason to keep EM in the package, rather than assume a clean sterility test covers it, is set out in FDA’s August 13, 2026 warning letter to Eugia Pharma Specialities. FDA wrote that a sterility test, “while a critical quality control for aseptically processed products that purport to be sterile, cannot be solely relied upon as justification to release drug product batches.”13 The same letter cites inadequate environmental monitoring of classified areas, including extended ISO 5 areas where critical aseptic connections occurred that were not held to ISO 5 limits.13 For more on making EM data usable for this kind of review, see our article on environmental monitoring data quality.

For non-sterile products, Annex 1 does not apply, but many sites still monitor classified or controlled areas. If the site’s own procedures say EM results are considered at release, the scorecard should include the same checks at a lower type. If the procedures do not say that, the scorecard should not invent a requirement.

Checks for Supplier CoAs and Equipment Status

Supplier CoAs and Material Status

Material data reaches the release package indirectly. The reviewer is not re-releasing each raw material, but needs confidence that every material lot used in the batch was released for use correctly at the time it was used. Under 21 CFR 211.84, each lot of components, containers, and closures must be withheld from use until it has been sampled, tested or examined, and released by the quality control unit. At least one test must verify the identity of each component. A supplier’s report of analysis may be accepted in place of other testing only if the manufacturer establishes the reliability of the supplier’s analyses through appropriate validation of the supplier’s results at appropriate intervals.14 Annex 16 points 1.7.2 and 1.7.6 expect the supply chain to be documented and the source and specifications of starting and packaging materials to comply with the marketing authorization.1

  • Every material lot consumed in the batch record matches a lot with released status at the time of dispensing (blocker). This is a reconciliation between the batch record and the inventory or LIMS status history, not a check of current status.
  • An identity test result is on file for each component lot (blocker).
  • Where the supplier’s CoA was relied on, the supplier was approved for that material at receipt and its reliability had been established within the defined interval (defect, or blocker for high-risk materials).
  • The source site on the CoA matches the approved source in the filing (blocker).
  • Any material released under a deviation or concession is linked to that deviation in the package (blocker).

The supplier reliability check is the one most often treated as a paperwork step. FDA’s June 2, 2026 warning letter to PLZ Corp, based on a records review under section 704(a)(4) of the FD&C Act rather than an inspection, concerned talc components tested for asbestos by the supplier and not by the manufacturer, with no records showing the manufacturer had established the reliability of the supplier’s analyses. FDA wrote that reliance on a CoA under 211.84(d)(2) “is permissible only if the drug product manufacturer establishes the reliability of the supplier’s test results through appropriate validation of the test results at appropriate intervals.”15 Our earlier article on supplier certificate of analysis data quality covers how to get CoA data into your systems cleanly. The release scorecard only needs to confirm that the result of that work exists for the lots used.

Equipment Status for the Batch Window

Equipment data reaches the reviewer in the least convenient form of any domain: logbooks, calibration certificates, and maintenance work orders, often in three systems. 21 CFR 211.182 requires a written record of major equipment cleaning, maintenance, and use, in individual equipment logs that show the date, time, product, and lot number of each batch processed, with entries in chronological order.16 21 CFR 211.67 requires equipment to be cleaned and maintained at appropriate intervals, with procedures that include inspection of equipment for cleanliness immediately before use.17 Annex 15 section 4 says equipment, facilities, utilities, and systems should be evaluated at an appropriate frequency to confirm they remain in a state of control.18 Annex 16 point 1.7.12 asks the QP to ensure that manufacturing and testing processes remain in the validated state.1

  • Every piece of major equipment named in the batch record was in qualified status on the dates it was used (blocker).
  • Calibration of critical instruments was in date on the dates of use, including laboratory instruments used for release tests (blocker).
  • The cleaning record for each piece of equipment is dated before its use for this batch, and the equipment log shows entries in chronological order (defect, rising to blocker where a cleaning record is dated after use).
  • Open maintenance work orders or out-of-tolerance calibration findings on equipment used during the window have been assessed for impact (defect, rising to blocker where an out-of-tolerance finding covers the window).
  • Any alarm or interruption recorded by equipment during the batch is accounted for in the batch record or a deviation (defect).

Status records can be wrong in a way that no data check will catch, and the scorecard should not pretend otherwise. In a January 7, 2026 warning letter to Winder Laboratories, FDA described manufacturing equipment, including a tablet press, observed with drug product residue and oxidized metal surfaces, “which were documented as being clean and were released by your quality unit.”19 A scorecard can confirm that a cleaning record exists and precedes use. It cannot confirm that the cleaning worked. That limit belongs in the scorecard procedure in writing, so no one mistakes a clean data check for a clean machine.

Cross-Cutting Checks

The remaining Annex 16 points apply to every batch of a product and change slowly. Most sites handle them best as a product-level status that the package references.

Annex 16 point 1.7.15

Changes Affecting the Batch

Every change control that touches the product, process, materials, equipment, or test methods used for this batch has an evaluated impact, and any extra checks or tests it required are complete.

Annex 16 point 1.7.17

Open Complaints and Recalls

Open complaints, investigations, and recalls for the product have been reviewed, with a statement that none of them negates the conditions for releasing this batch.

Annex 16 point 1.7.14

Stability and Commitments

Ongoing stability data still supports certification, and any post-approval commitments on manufacture or testing have been addressed.

Annex 16 point 1.7.18

Technical Agreements

Required technical or quality agreements with contract manufacturers and laboratories that touched the batch are in place and in date.

The complaint check is the one most likely to be skipped, because complaint data lives far from release. FDA’s April 30, 2026 warning letter to CareFusion 213 described more than 2,500 customer complaints about its drug products since September 2023, including reports of foreign matter and suspected mold, with investigations that repeatedly failed to identify root causes or extend to other potentially affected batches.20 A release reviewer who cannot see open complaint investigations for the product is missing data Annex 16 expects them to weigh.

Scoring the Package Before It Reaches the Reviewer

The checks above become a scorecard once each one has a type, a weight, and a rule for what happens when it fails. The design choice that matters most is the order: gate first, score second.

Three Types of Check

Weight 5

Blocker

A failed blocker means the package is not ready for review. It goes back to the record owner. No score, however high, overrides an open blocker. Examples: an unlinked deviation, a missing release test, equipment out of calibration during use.

Weight 2

Defect

A failed defect does not stop the package, but it must be corrected or explained before handoff. Defects count against the readiness score. Examples: a correction without a reason, an unsigned trend review.

Weight 0

Observation

An observation is recorded for the reviewer and for trending, but does not count against the score. Examples: pending EM results with expected dates, a note that a CAPA remains open.

Excluded

Not Applicable

A check that does not apply to this product or batch is removed from the denominator, with the reason recorded. EM checks for a non-sterile product with no classified areas are a common case.

The Scoring Rule

The readiness score is calculated at the first pre-check, before any corrections. It measures how good the package was when the assembler first said it was ready. The formula is:

Readiness score = 100 × (1 − weighted failures ÷ weighted applicable checks)

The package’s status is decided separately. If any blocker failed, the status is “not ready” and the package goes back. If no blocker failed, defects are corrected or explained, and the package goes to the reviewer with its score and its list of corrections attached. Scoring at the first pre-check, not after correction, matters. A score taken after correction would read close to 100 for every package and tell you nothing about where the rework comes from.

A Sample Scored Package

The table below shows a scorecard for a hypothetical sterile injectable batch. It is a completed example of the method, not data from any real site. The 22 checks are drawn from the lists above.

IDDomainCheckTypeWeightFirst Pre-Check Result
BR-1Batch recordAll steps executed and signed, no blank required fieldsBlocker5Pass
BR-2Batch recordCorrections follow good documentation practice, reason recordedDefect2Fail: two corrections with no reason
BR-3Batch recordYield and reconciliation within limits or linked to a deviationBlocker5Pass
BR-4Batch recordLabel reconciliation complete and within limitsDefect2Pass
DV-1DeviationsEvery deviation in the batch window or on shared resources linkedBlocker5Pass
DV-2DeviationsEach linked deviation has an approved impact assessment for this batchBlocker5Fail: one assessment drafted, not approved
DV-3DeviationsInvestigation scope stated with reason for batches included or excludedDefect2Pass
TR-1Test resultsEvery test on the approved specification has a reported resultBlocker5Pass
TR-2Test resultsSpecification version matches the approved applicationBlocker5Pass
TR-3Test resultsAny OOS, OOT, or retest covered by a concluded investigationBlocker5Pass
TR-4Test resultsSecond-person review of laboratory records completeDefect2Pass
EM-1Environmental monitoringBatch window mapped; all scheduled results presentBlocker5Pass
EM-2Environmental monitoringEvery action limit excursion linked with an impact conclusionBlocker5Pass
EM-3Environmental monitoringAlert results and trends reviewed, review signedDefect2Fail: trend review not signed
SM-1MaterialsEvery lot consumed had released status at time of dispensingBlocker5Pass
SM-2MaterialsSupplier approved at receipt; CoA reliability established in intervalDefect2Pass
SM-3MaterialsIdentity test result on file for each component lotBlocker5Pass
EQ-1EquipmentEquipment qualified and instruments calibrated on dates of useBlocker5Pass
EQ-2EquipmentCleaning records precede use; logs in chronological orderDefect2Pass
EQ-3EquipmentOpen work orders and out-of-tolerance findings assessedDefect2Pass
X-1Cross-cuttingChanges affecting the batch evaluated, required checks completeBlocker5Pass
X-2Cross-cuttingOpen complaints and investigations for the product reviewedDefect2Pass
Totals833 failures, weighted 9 (one blocker)

The 22 checks include 13 blockers at weight 5 (65 points) and 9 defects at weight 2 (18 points), for 83 weighted points. The three failures are BR-2 (2 points), DV-2 (5 points), and EM-3 (2 points), a total of 9. The readiness score is 100 × (1 − 9 ÷ 83), which rounds to 89.

An 89 sounds good. The package is still not ready, because DV-2 is a blocker. This is the point of gating first. A site that sent packages to the reviewer whenever the score passed some threshold would have sent this one, and the reviewer would have found an unapproved impact assessment, returned the package, and lost a day. Under the gate rule, the deviation owner gets the impact assessment approved, the assembler fixes the two corrections and gets the trend review signed, and the package goes to the reviewer once, complete.

What the score is for. The readiness score is not a release criterion. It is a measure of how much rework the package needed before it was fit to review, and it is useful mainly in aggregate. When the average first-pass score for a product line drops, or when one domain accounts for most of the failed points, that tells the site where to fix its upstream process.

When and How the Pre-Check Runs

1

Assembly Opens at Batch Completion

The assembler opens the package when the last packaging step is complete and starts pulling records from each source system, using a fixed list for the product.

2

System Checks Run First

Checks that can be answered from system data, such as material lot status at time of use, calibration dates, and specification versions, run automatically or by query where the systems allow it.

3

The First Pre-Check Is Scored

The pre-checker completes the remaining checks and records the first-pass result for every check. This is the number that feeds the readiness score.

4

Gaps Return to Record Owners

Failed blockers and defects go back to the owner of the source record, not to the assembler, with a reason code from the scorecard.

5

Handoff With the Scorecard Attached

When no blocker is open, the package goes to the reviewer with the scorecard, the list of corrections, and the open observations visible on the first page.

Tracking Right-First-Time Release

The readiness score measures the package before the reviewer sees it. Right-first-time release measures what happens after. Both are needed, and they must not be confused.

Defining Right-First-Time Release Precisely

We define right-first-time (RFT) release as the share of release packages that reach a disposition decision, release or reject, without the reviewer returning them for correction or for missing information. A package that the reviewer rejects because the batch failed is still right first time from a data point of view: the data was complete and the decision was made. A package that the reviewer sends back because a deviation assessment is missing is not, even if the batch is eventually released.

This is a different measure from manufacturing RFT, which counts batches made without deviations, and from the lot acceptance rate that FDA proposed in its 2016 revised draft guidance on submitting quality metrics data. That draft defined lot acceptance rate as the number of accepted lots in a timeframe divided by the number of lots started by the same establishment in that timeframe.21 Lot acceptance rate measures process outcomes. RFT release measures data readiness. A site can have a high lot acceptance rate and a poor RFT release rate, which usually means good batches are waiting in quarantine for paperwork.

Status of FDA’s quality metrics program. FDA has not put a quality metrics reporting requirement in place. Its quality metrics page describes the agency as proposing to develop and implement a Quality Metrics Reporting Program, notes a March 2022 docket seeking comment on changes to the program, and lists the November 2016 revised draft guidance among historical resources.22 Sites should build RFT release because it is useful to them, not because they expect to report it.

Return Reason Codes

Every return from the reviewer should carry a reason code tied to the same domains and check IDs as the scorecard. This is what connects the two measures. If the reviewer returns a package for an unlinked deviation, the code is DV-1. Over a quarter, the return codes show which checks the pre-check is failing to catch.

Keep the code list short. A code list with fifty entries will be used inconsistently. The scorecard’s own check IDs, plus a small number of codes for things outside the scorecard, such as “reviewer requested additional justification” and “new information arrived after handoff,” are enough.

The Measures to Track

MeasureDefinitionWhat It Tells You
First-pass readiness scoreAverage readiness score at first pre-check, by product and by domainWhere upstream records are weakest when packages are first assembled
Blocker rate at pre-checkShare of packages with one or more failed blockers at first pre-checkHow often packages are declared ready when they are not
RFT release rateShare of packages reaching a disposition decision with no reviewer returnWhether the reviewer’s time goes to judgment or to chasing
Pre-check escape rateShare of reviewer returns whose reason code matches a scorecard check that was marked passWhether the pre-check itself is reliable
Time in each stateDays from batch completion to assembly complete, to handoff, to decisionWhere release time is spent: assembly, rework, or review
Age of open items at assemblyDays that each open deviation, investigation, or pending result has been open when the package is assembledWhich upstream processes hold release, and for how long

The pre-check escape rate is the measure most programs leave out, and it is the one that tells you whether the rest can be trusted. If the pre-checker marks DV-2 as passed and the reviewer then returns the package for an unapproved impact assessment, that is an escape. A rising escape rate means the pre-check is turning into a formality, which is the worst outcome: a scorecard that looks reassuring and is not.

What the Published Benchmarks Say

Published benchmarks for release package quality are thin, and the ones that exist are dated and come from practitioner articles rather than formal studies. A 2012 BioPharm International article by Simon Chalk described a one-day sign-off for batch records as a best practice in the industry and said that achieving it means 95% of batch records should be right first time. The same article described review lead times falling from tens of days, with 30 common, to single figures at companies that worked on the problem.8 A 2019 BioPharm International article by Johan Zebib on review by exception stated that “typical paper-based batch release times can take 10–40 days.”7

Those figures describe batch record review, which is one part of the release package, and they predate the current Annex 1. They are reasonable reference points for ambition, not targets to copy. A better approach is to measure your own baseline for a quarter before setting any target, and then set targets by product family, since a complex sterile biologic and a simple oral solid will never have the same package.

Keeping the Scorecard Honest Over Time

A release package scorecard is easy to start and easy to let decay. The failure patterns are predictable, and most can be designed out at the start.

Do Not Let the Pre-Check Become the Review

The pre-check confirms that data is present, linked, and consistent. The review decides whether the batch should be released. When a site gets comfortable with its scorecard, teams are tempted to let the reviewer rely on it completely and look only at exceptions. That can be done properly, but only with the same discipline that any review-by-exception approach needs: validated logic, documented rationale for what the reviewer does not look at, and evidence that the checks still work. Without that, the scorecard becomes a reason to stop looking. Annex 16 keeps the QP’s reliance conditional on “on-going assurance.”1 The escape rate is the evidence for that assurance.

Do Not Turn the Score Into a Production Target

Once a readiness score exists, someone will want to put it on a production dashboard. Be careful. If production teams are judged on package scores, the easiest way to raise the score is to raise fewer deviations. That would be a serious harm to the quality system. Report the score to the people who own the records that fail, by check ID, as information for fixing the upstream process. Do not report it as a performance grade for individuals or shifts.

Review the Check List When Things Change

The scorecard’s checks must change when the product, the specification, the systems, or the regulations change. A new specification test that is not added to TR-1’s check list will pass every pre-check and fail every review. Put the scorecard under the same change control as the release procedure it supports, and add a scorecard review step to the change control template for specification changes, new equipment, and system upgrades.

Bring the Measures Into Management Review

ICH Q10 says management review should give assurance that process performance and product quality are managed over the lifecycle, and that the periodic review of the pharmaceutical quality system should assess performance indicators, including for complaint, deviation, CAPA, and change management processes.23 RFT release, the escape rate, and the age of open items at assembly fit naturally into that review. They show, in a way leadership can follow, whether the quality system is producing release-ready data or whether reviewers are fixing the gaps by hand.

A Staged Way to Start

1

Baseline the Returns

For one product family, log every reviewer return for six to eight weeks with a reason. This becomes the first draft of the check list and the baseline for RFT release.

2

Build the Blockers First

Write and type the checks that would have caught the most common returns. Start with blockers only, and run them manually.

3

Add Defects and Scoring

Once the blocker gate is stable, add defect checks and begin recording first-pass scores and return codes against check IDs.

4

Automate the Data Checks

Move the checks that can be answered from system data into queries or system rules, starting with material status at time of use, calibration dates, and specification versions.

5

Extend and Calibrate

Extend to further product families. Hold a short calibration session each quarter in which pre-checkers and reviewers score the same package and compare results.

Sites that try to start with all six domains, automated, across all products, tend to spend months arguing about the check list and never run it. A site that starts with the returns its reviewers already complain about will have a working gate within weeks, and the data from that gate will tell it where to go next.

Conclusion

Batch release is where every upstream data problem shows up, and it is where a site can least afford to discover them. The reviewer, whether a QP certifying under Annex 16 or a quality unit deciding disposition under 211.192 and 211.165, depends on a package of records that comes from several systems and several teams. Measuring that package as a unit, with checks drawn directly from the requirements, gated on blockers, scored before the reviewer sees it, and tracked through right-first-time release and the pre-check escape rate, turns “the reviewer will catch it” into a process the site can see and improve. The enforcement record shows what happens without it: batches released with open investigations, on passing retests, or on supplier data no one had confirmed.

Sakara Digital works with pharma and biotech organizations on the data foundations behind quality decisions, including the links between batch records, deviation systems, LIMS, and equipment records that a release package depends on. If you are looking at your release process and want an independent view of where the data gaps are and which checks to build first, we are happy to have that conversation.

For Further Reading