Data Catalog Selection for Life Sciences: Beyond the Feature Grid

What a Feature Grid Tells You

Ask three catalog vendors for a capability matrix and you will get three documents that are roughly ninety percent identical. Automated technical metadata harvesting from cloud warehouses, relational databases, file stores, and business intelligence tools. A business glossary with term-to-asset linking. Lineage visualization. Data quality rule integration. Classification and sensitivity tagging. Access request workflow. An application programming interface. A browser extension or plugin that surfaces catalog context inside other tools. Increasingly, a natural language search assistant that answers questions about the organization’s data.

Every one of those rows will show a check mark. The differences that matter are hidden inside the check marks, and the demo is specifically designed not to reveal them. This is not vendor dishonesty. It is what a demo is for. The demo runs on a prepared dataset with clean schemas, meaningful column names, complete descriptions, and a lineage graph that resolves end to end because someone built the pipeline specifically so that it would resolve.

Why convergence happened

The catalog market matured. Capabilities that were differentiators in 2018 became table stakes. Open source projects published their lineage parsers, which pushed commercial products to match them. Cloud warehouse providers built native cataloging into their platforms, which compressed the space in which a standalone product could claim a unique technical capability. The result is that the feature list is now a description of a category, not a description of a product.

What still differs between products does not show up in the feature list: the quality of the automated inference, the friction of the curation interface, the honesty of the lineage rendering, the relevance of the search ranking, and the depth of integration into the tools where people work. None of those five things appears as a row on a matrix, because none of them is a yes or no answer. They are all questions of degree, and degree can only be measured against your own data.

What the demo optimizes for

A vendor demo is an argument that the product is easy. It is staged in an environment where the hard part has already been done. When a presenter clicks into a table and shows a rich description, a clear business owner, a set of glossary terms, and a quality score, the natural inference is that the product produced all of that. In almost every case, a human produced most of it, and the product stored it.

The right response is not cynicism. It is a specific question: which parts of this screen were generated without human input, and which parts required someone to type them? Ask the presenter to open a table that nobody has curated. Most will not have one available. That reluctance is itself informative.

The failure mode this produces

Selection on features produces a predictable outcome. The organization buys a capable product, connects it to every source it can reach, and celebrates a catalog containing hundreds of thousands of assets within a quarter. Coverage becomes the reported metric. Then the second year arrives, the implementation partner leaves, the stewardship model turns out to be two people with other full-time jobs, and the descriptions never get written. The catalog is complete and useless at the same time. Users search, find nothing they recognize, and stop searching. Trust, once lost, is much harder to rebuild than it was to establish.

Curation Economics: The Argument That Should Drive the Decision

Here is the central claim of this article. A data catalog does not fail on technology. It fails on curation economics. Automated harvesting is cheap and produces technical metadata that almost nobody needs. The business context that makes a catalog valuable is expensive, requires people with domain knowledge, and is almost never budgeted. The gap between what the machine can produce and what a user needs is the entire problem, and every meaningful selection criterion is a restatement of it.

Harvesting produces volume, not meaning

Connect a catalog to a clinical data warehouse and it will return, within hours, a complete inventory of schemas, tables, views, columns, data types, nullability, row counts, and update timestamps. That is real and it has value. It is also information that any competent person could obtain by querying the information schema.

What a user needs to know is different in kind. Is this the dataset the biostatistics group used for the primary analysis, or the one the data management group used for cleaning? Does this column contain the value as collected or the value after the derivation rules were applied? Was this table built for a specific study and abandoned, or is it maintained? If two tables both look like adverse event data, which one is the one people trust, and why? None of that is present in a schema. All of it is in someone’s head.

Why the useful context is expensive

The reason business context does not get written is not laziness. It is that writing it correctly requires someone who understands both the data and the science, and that person is the scarcest resource in the organization. A clinical data manager who can explain why a derived variable in an analysis dataset differs from its source variable is also the person the study team needs for six other things this week. Asking that person to spend two hours a week writing catalog descriptions is asking for something real, and if you do not plan for it, it does not happen.

The published research on making life sciences data findable and reusable is consistent about this. Work on selecting datasets for FAIRification in drug discovery and development notes that the tractability of any such effort depends on the skills, competencies, resources, and time available, and identifies personnel and the long-term sustainability of data resources as the binding constraints. The same work found that selection criteria are frequently opaque and inconsistent, and describes prioritizing 15 projects out of more than 120 as a deliberate act of triage rather than a comprehensive program.³ That is what curation at scale looks like in practice: choosing what not to do.

The Pistoia Alliance bioassay annotation work is a useful illustration of the alternative pattern. That project used a hybrid approach in which natural language processing produced initial extraction and automated annotation suggestions, and human experts then reviewed them. It annotated 2,353 assay protocols drawn from several sources, with roughly 2,800 previously published assay methods covered overall, and reported that commercial assay panels were the easiest to annotate while protocols from peer-reviewed papers were the most difficult.¹⁴ The lesson for catalog selection is not that automation solves the problem. It is that automation plus a cheap human review step solves more of the problem than either alone, and the review step has to be cheap.

Two numbers to compute before you look at a product

Before any vendor conversation, calculate two things about your own environment. They will do more to shape the decision than any comparison document.

Steward capacity per year. Count the people who will realistically curate, multiply by the hours per week they will give, and multiply by 45 working weeks. In most mid-size biotech and pharma settings the honest answer is somewhere between 200 and 800 person-hours per year across the whole organization. That is your entire manual curation budget.

Assets that need context. Not total assets. The subset that a human will plausibly search for and need to understand. In a clinical data environment this is usually study-level datasets, analysis datasets, standard domains, and the small number of derived tables that feed reporting. In manufacturing it is batch records, process parameter tables, and the master data that ties them together. This number is typically one to five percent of the harvested inventory.

Divide the first by the second. If the answer is under ten minutes per asset, no amount of stewardship discipline will save a product that requires a human to write everything from scratch. You need a tool whose unattended output already carries meaning, and whose curation interface lets a steward correct or extend that output in under two minutes.

Generative description features deserve a specific test

Most products now offer automatically generated descriptions from a large language model. These are worth having and they are also the easiest place for a demo to mislead. A generated description that says “this table contains subject visit data” adds nothing. A generated description that correctly identifies a column as a derived study day relative to first dose, and flags that the derivation differs from the standard definition, adds a great deal.

Test the feature on your least documented, most idiosyncratic tables. Then have a domain expert grade a sample of 50 generated descriptions on a three-point scale: useful, harmless, or wrong. The wrong category is the one that matters. A catalog full of confidently phrased incorrect descriptions is worse than an empty one, because it teaches users that the catalog cannot be trusted, and it does so faster than blank fields would.

The Life Sciences Requirements Generic Guidance Misses

Generic catalog guidance is written for a world in which the worst outcome of using the wrong dataset is a bad decision. In pharma and biotech, the worst outcome is a regulatory finding, a compromised submission, or a use of participant data that the participant never agreed to. That difference changes what has to be on a catalog entry.

Validation status and record classification belong on the entry

A user who finds a dataset in the catalog needs to know, before doing anything with it, whether it comes from a validated system, whether it constitutes a regulated record, and what its retention obligations are. Not in a separate register that a governance team maintains in a spreadsheet. On the entry, visible, and derived from a source of truth rather than typed by hand.

The regulatory basis for this is not new. The MHRA data integrity guidance defines metadata as data that describe the attributes of other data and provide context and meaning, including structure, data elements, interrelationships, and audit trails, and treats metadata as an integral part of the original record. EU Annex 11 and 21 CFR Part 11 both establish expectations for the reliability and traceability of electronic records in regulated processes. ICH E6(R3) makes data governance an explicit shared responsibility of sponsor and investigator, with clarified expectations for metadata, audit trails, and traceability across the data lifecycle. The point is not that these documents mandate a catalog. It is that they define the attributes a catalog entry has to carry if the catalog is going to be used for anything that touches a regulated process.

Ask a candidate product three concrete questions. Can a custom attribute be defined at the asset level, populated from an external system through the interface, and displayed prominently rather than buried in a properties panel? Can that attribute drive search filtering and access policy, so that a user can restrict a search to validated sources only? And is the attribute versioned, so that when a system’s validation state changes, the catalog records when it changed rather than overwriting it without a record?

Controlled vocabularies and reference data need first-class treatment

In most industries, reference data is a set of small lookup tables that nobody thinks about. In life sciences it is a versioned, externally governed, regulator-facing body of content that changes on a published schedule and whose version determines whether a submission is acceptable.

CDISC Controlled Terminology is the set of codelists and valid values used with data items in CDISC-defined datasets, and it provides the values required for submission to FDA and PMDA in compliant datasets. It is developed and maintained in collaboration with the National Cancer Institute Enterprise Vocabulary Services, which supplies the semantic infrastructure behind it and the mappings that connect it to other standards.¹⁰¹¹ On the product and regulatory side, the ISO IDMP standards define how medicinal products, substances, dose forms, units, and packaging are identified and described, and EMA implements them through a master data framework covering substance, product, organization, and referential domains.¹² In observational research, the OMOP common data model and its standardized vocabularies play a similar role.¹⁹ In the laboratory, the Allotrope framework combines a data format, data models, and an ontology suite that provides a standard vocabulary and semantic model for analytical processes across equipment, material, process, and results.¹³

A catalog that treats all of this as ordinary tables will not help. What first-class treatment means in practice:

• The vocabulary itself is a catalog object, with its version, publication date, source authority, and change history, not just a table containing its current contents.
• Assets declare which version they conform to. A dataset built against one terminology release and a dataset built against a later one are not interchangeable, and the catalog should make that visible without anyone having to open the data.
• Term-level linking works in both directions. A user looking at a coded column can see the codelist; a user looking at a codelist can see every asset that uses it. This is what makes impact analysis possible when a terminology release changes a preferred term.
• External ontologies can be imported or referenced rather than retyped. Resources such as the EMBL-EBI Ontology Lookup Service exist precisely so that organizations do not maintain private copies of public vocabularies.²⁰ A catalog that can only hold a flat internal glossary will force you to maintain those copies.

This is the requirement most often missing from generic evaluations, and it is the one that most reliably separates products in a real trial. Ask to model a real codelist with two versions and an asset that conforms to the older one. Watch how much of that has to be done by hand.

Consent and use limitations must travel with the asset

Clinical and human biological data carries conditions that are set at collection and constrain everything afterward. The consent a participant gave was for a defined purpose. Reuse for a different purpose may be permitted, may require a fresh approval, or may be prohibited outright, and the answer depends on the consent language, the protocol, the applicable law, and any contractual terms attached to the study.

The literature on sharing individual participant data is direct about the limits here. Because the nature, purpose, and destination of future data sharing cannot be predicted at the time of enrollment, any consent for secondary use cannot be fully informed, which is why broad consent with a scientific-purpose restriction became the working compromise.⁷ That compromise only functions if the resulting conditions are recorded somewhere a future user will see.

There is a mature standard for exactly this. The Data Use Ontology, developed through the Global Alliance for Genomics and Health, is a hierarchical vocabulary of human and machine readable terms that represents a dataset’s allowable uses. As of June 2021 it contained 25 terms across two types, permissions and modifiers, and over 200,000 datasets worldwide had been annotated with machine readable DUO terms.⁴ It is distributed as a versioned, publicly available file written in the W3C Web Ontology Language and following open biomedical ontology development principles, and it is implemented by major genomic data holders internationally.⁵ Companion guidance addresses how to write consent forms that map directly onto those terms.⁶

Regulatory direction is moving the same way. The European Health Data Space Regulation entered into force in March 2025 and applies in stages under its own transitional article, establishing an EU framework for both primary and secondary use of electronic health data.⁸ Analysis of the secondary use regime highlights the practical dependency on dataset descriptions being discoverable and on metadata being expressive enough to carry real constraints, and notes that current metadata formats have limits in how precisely they can express restrictions on what may be used.⁹ A catalog that cannot represent a use condition at all will not be adequate for that environment.

The evaluation question is specific: can the catalog hold structured use conditions, propagate them to derived assets, surface them in search results rather than only on the detail page, and block or warn on an access request that conflicts with them? Free-text notes do not qualify. A note is not a control.

Where FAIR helps and where it does not

Most life sciences organizations already have some commitment to the FAIR principles: findable, accessible, interoperable, reusable.¹ Work on FAIR in biopharmaceutical research and development has made the case that the payoff comes from letting analytical tools reach data automatically at scale, not just from helping humans browse.² A catalog is the most visible instrument for the findable part, and it contributes to reusable through provenance and license or use terms.

What FAIR does not do is tell you which product to buy, and treating a catalog purchase as a FAIR program is a category error that shows up repeatedly. FAIR is a set of qualities the data should have. The catalog is one mechanism among several. If the selection conversation slides into a discussion of FAIR maturity rather than curation effort and search relevance, it has drifted away from the decision at hand.

Lineage: Everyone Claims It, Few Can Show It Honestly

Lineage is the feature most likely to be oversold and least likely to be tested properly during selection. Every product claims it. The claim is technically true in every case. The claims are not comparable.

Column level and table level are different products

Table-level lineage says that table B was built from table A. Column-level lineage says that column B.study_day was derived from A.visit_date and A.first_dose_date. The difference in usefulness is enormous, and the difference in engineering difficulty explains why so many products stopped at the first.

Table-level lineage can often be captured from orchestration logs alone. Column-level lineage requires parsing the transformation logic itself: the SQL, the Python, the Spark job, the transformation framework model. That is substantially harder, which is why many products historically supported only table-level lineage, since extracting referenced tables from a query is far easier than resolving individual column derivations.¹⁵

For a regulated organization, the distinction is not academic. If you cannot say which source fields produced a reported value, you cannot use the lineage graph to answer an inspector’s question about how that value was derived. You are back to reading code.

What breaks the parser

Column-level lineage is produced by a parser, and parsers fail in specific, knowable ways. Published accounts from the teams that build these parsers are unusually candid about the failure modes. Parsers that lack awareness of the underlying table schemas cannot generate accurate column-level lineage, because a select-star expansion is unresolvable without knowing the columns. Dialect coverage is uneven, and general-purpose query planners are missing support for widely used warehouse dialects. Errors occur when parsing fails outright or when a column’s origin cannot be determined, as with JSON unpacking or lateral joins, and in those cases the lineage is simply incomplete. Common table expressions add another layer, since they can be nested, can contain select-star, and require recursive traversal to resolve a real origin.¹⁵¹⁷ Transformation frameworks that generate lineage from their own compiled models will also carry documented caveats about which constructs are covered.¹⁶

Now add the life sciences environment. Derivations that live in statistical programming code rather than SQL. Data that moves between systems through validated file transfers with no queryable log. Vendor systems whose internal processing is not visible at all. Spreadsheets in the middle of a chain. Manual reconciliation steps that exist precisely because two systems disagree. Every one of these is a place where an automated lineage graph goes dark.

How to test lineage honestly

Pick one reported value that matters and that you already understand well enough to reconstruct by hand. A key efficacy summary figure, a batch release parameter, a safety signal count. Then, during the proof of concept, ask the catalog to show you its provenance and compare its answer against the truth you already know.

Score three things. Completeness: what fraction of the real chain appears. Accuracy: whether any edge it draws is wrong. Honesty: whether the places it could not resolve are marked as unresolved or dropped without any marker. The third score is the one to weight most heavily, and it is the one that never appears in a matrix.

Then ask the operational question that follows. When lineage is incomplete, what is the supported way to record the missing edge manually, who can do it, does it look visually distinct from automatically derived lineage, and does it survive the next harvest? A catalog that cannot accept human-declared lineage will always understate provenance in an environment where some processing is not machine-readable, which is every life sciences environment.

Adoption: A Catalog Nobody Searches Is Dead

All of the preceding is irrelevant if people do not use the thing. Adoption is usually treated as a change management problem to be handled after selection, through training and communication. That framing is wrong. Adoption is largely determined by two product characteristics that can be evaluated before purchase: whether search works on the words users type, and whether the catalog reaches people inside the tools they already have open.

Search has to work on your vocabulary, not the vendor’s

A scientist does not search for “SDTM AE domain.” They search for “adverse events study 204.” A manufacturing engineer does not search for “batch_genealogy_fact.” They search for the product name and the word “yield.” A commercial analyst searches for a brand name that appears in no table name anywhere in the warehouse.

The gap between user vocabulary and technical object names is the single biggest determinant of whether search returns anything useful, and it is the thing a prepared demo hides most effectively, because the demo dataset was named sensibly. Research on dataset discovery has found that users struggle to convey implicit preferences and lack visibility into the search space and the criteria by which results are included or excluded, and that assessing whether a dataset suits a task requires understanding its granularity, attributes, size, the goals it was created for, and its relevance to the work at hand. Keyword and semantic approaches both fall short of that, which is why users need interactive guidance and transparency rather than a ranked list.¹⁸

The practical test is straightforward and rarely run. Before the proof of concept, collect 30 to 50 real search phrases from the people who will use the catalog. Ask them what they went looking for last month and write down their exact words, including the abbreviations and the study codes and the misspellings. Run those phrases against each candidate product with your own data loaded and no manual tuning. Record whether the right asset appears in the top five results.

Then run them again after a steward has spent one hour adding synonyms and glossary terms. The improvement between the two runs tells you how much curation effort search relevance requires, which feeds straight back into the curation economics calculation.

Reaching people inside their own tools

The other determinant is whether the catalog is a place users have to go or a layer that comes to them. A separate portal that requires a separate login, opened deliberately, competes with the far cheaper alternative of asking a colleague. Most people will ask the colleague. Programs consistently fail not because the technology was poor but because the tools were outside daily work and people ignored it.

What matters is whether catalog context appears in the notebook, the query editor, the business intelligence tool, the chat channel, and the electronic laboratory notebook. A hover over a column name that shows the description and the use conditions. A bot that answers a data question in the channel where the question was asked. A warning inside the query tool when someone selects a deprecated table.

Evaluate this concretely. List the five tools your target users spend the most time in. For each, ask what the integration does: whether it is read-only or supports curation in place, whether it requires an administrator to install per user, and whether it works with your authentication setup. Vendors will claim integration with all five. The depth varies from a fully embedded panel to a link that opens the portal in a new tab.

The stewardship model is part of the product decision

Products embed assumptions about who curates. Some assume a central team of professional stewards working through a queue. Some assume distributed domain ownership where the people closest to the data document it in place. Some assume curation is mostly automatic and humans only correct errors.

None of these is wrong. The wrong one is the one that does not match your organization. If your data governance model is federated, a product built around a central curation queue will fight you constantly. If you have a small central team and no realistic path to distributed ownership, a product that depends on domain experts contributing voluntarily will produce an empty catalog. Match the assumption to the reality you have, not the one described in your target operating model.

Running the Evaluation on Your Own Messiest Domain

The single most valuable change you can make to a catalog selection is to stop evaluating on the vendor’s dataset and start evaluating on your worst one. Not a representative sample. The domain everyone complains about: the one with inconsistent naming, undocumented legacy tables, three overlapping sources of the same fact, and processing steps that live in code nobody owns. If a catalog helps there, it will help everywhere. If it only works on the tidy domain, you have learned nothing, because the tidy domain did not need a catalog.

What the question set should contain

Balance the question set across the types of work the catalog is meant to support. A useful distribution for a life sciences organization looks roughly like this: five questions about finding a dataset by describing it in business language, four about understanding what a specific field means and how it was derived, three about determining whether an asset may be used for a stated purpose, three about who owns something and whether it is still maintained, three about impact if a source changes, and three about which of several similar assets is the authoritative one.

Grade each answer on a simple scale: answered correctly and quickly, answered but required interpretation, or not answered. Do not let a vendor answer on the catalog’s behalf during the session. If a person from the vendor has to explain where to click, the answer is “required interpretation” at best, because your users will not have that person.

What to watch for during the trial

Three signals during a proof of concept are more predictive than the formal scoring, and all three are behavioral rather than technical.

• How the vendor reacts to a failure. Every product will fail something in a real environment. A vendor who explains the limitation precisely, says what is on the roadmap and what is not, and does not reframe the failure as a configuration issue is telling you what support will be like. A vendor who insists the problem is your data is telling you something too.
• Whether your own people want to keep using it. At the end of the trial, ask the two stewards whether they would rather keep this tool or go back to what they had. Ask the same of three users. Enthusiasm from people who were not part of the selection committee is the strongest signal available.
• How much of the work the vendor did. Track it honestly. If the vendor’s team spent 60 hours making the trial succeed, that effort does not disappear after purchase, it just moves onto your budget or your backlog.

Contracting and the Questions Vendors Would Rather Skip

Once a product is chosen, a few contractual and commercial questions determine whether year three looks like year one. They are easy to raise while there is still competitive tension and nearly impossible to raise afterward.

Get your curation out. The descriptions, glossary terms, ownership assignments, use conditions, and manual lineage your people write are the most valuable thing in the system and the only part that is yours alone. Confirm that a full export exists, that it includes human-authored content in a structured and documented format rather than a screen scrape, and that you can run it yourself without a support ticket. Test the export during the proof of concept. An export that only a vendor engineer can produce is not an export.

Understand how pricing scales with the thing you will do most. Some products price on assets, some on users, some on connectors, some on compute for harvesting. Model your realistic three-year state under each. Asset-based pricing in an environment that generates new tables continuously behaves very differently from user-based pricing, and the difference is rarely visible in a first-year quote.

Ask about the validated deployment path. If the catalog will hold or display information relied on in a regulated process, you need to know what the vendor supplies: qualification documentation, change notification lead time for a cloud service, the ability to defer an upgrade, audit access, and how their release cadence interacts with your change control. Vendors serving regulated industries will have answers. Vendors who have not thought about it will say the catalog is not a regulated system, which may be true today and is a decision you should make rather than inherit.

The organizational commitment that has to come with it

No product choice compensates for an absent stewardship model. Before purchase, three things should be settled and written down: who owns curation for each domain, how much of their time is committed and by whom, and what happens when they leave. The most common failure in catalog programs is the absence of defined ownership at go-live, and the consequence is a catalog that goes stale quickly and loses user trust faster than it can rebuild it.

Fund the first year of curation explicitly, as a named allocation of people’s time rather than an expectation. Pick a small, high-value scope for the first six months. Publish the adoption metrics openly, including the unflattering ones. A catalog program that reports coverage while usage declines is measuring the wrong thing on purpose.

Conclusion

Catalog products have converged enough that feature comparison no longer discriminates between them. What still discriminates is the relationship between what the tool produces on its own and what your people have to add by hand, because that relationship determines whether the catalog is alive in year three or a searchable inventory of table names that nobody trusts. Measure unattended yield, measure steward effort per asset, measure whether curation survives change, and measure whether context propagates. Those four numbers, taken on your own data, will tell you more than any scoring matrix.

The life sciences additions are not optional refinements. Validation status and record classification have to be visible on the entry. Controlled vocabularies and reference data have to be governed objects with versions, not lookup tables. Consent and use limitations have to travel with the asset in a structured form that can be acted on, because a catalog that makes restricted data findable without making its restrictions visible has made governance worse without anyone noticing. And lineage deserves more skepticism than it usually gets: a graph that hides its gaps is a confident claim about provenance that cannot be defended, and honest gaps are worth more than a clean picture.

Sakara Digital works with pharma and biotech organizations building the data foundations that catalogs are supposed to serve. If you are evaluating a data catalog and want an independent read on curation effort, life sciences requirements, or how to structure a proof of concept that tells you something real, we are happy to have that conversation.

For Further Reading