Every conversation about data catalogs seems to end the same way: we should be more disciplined about updating them. Assign owners. Run a documentation sprint. Make stewardship someone’s actual job.
So companies do. And eighteen months later the catalog is stale again, and the conclusion is that the team needs more discipline.
I want to argue the opposite. Your team can be rigorous and still end up with a stale catalog. Everything the catalog can measure without applying a definition is a proxy for the thing that matters: whether the definition still produces an answer the business recognizes as true. A catalog can tell you a definition is old. Real use can reveal that it is wrong: someone applies the definition, gets an answer that does not match reality, and says so. None of the fifteen catalogs we checked documents learning automatically from that moment, not even the ones that watch queries all day.
One definition, eighteen months
Take a definition almost every company has. Say someone wrote this into the catalog during a documentation sprint:
active_customer: a customer with at least one paid invoice
in the previous 30 days. Owner: analytics. Certified.Perfectly reasonable. Correct, even.
Then in March the company launches a free trial motion. Trials create accounts and show activity, but finance decides they do not count as active customers until they convert. The decision is made in a meeting, shared in a thread, and applied to two dashboards by the analyst who was in the room.
Nobody touches the catalog. Why would they? The page reads fine. It is certified. It has an owner.
The definition is now wrong, and nothing in the catalog knows it.
Catalogs did get better at detecting staleness
To be fair to the category, modern catalogs detect a lot. Atlan documents a freshness playbook you can set up. After 30 days without updates, an asset is marked stale. After 60, it is pending deprecation. After 90, its certification flips to deprecated. Purview’s data health checks flag a published glossary term whose description is shorter than 25 characters. Euno watches your dbt manifest for changes and can open pull requests against the BI repository. Databricks marks a table stale when a commit arrives unusually late.
Teams should use all of that. But look at what the checks run on: days since the last update, a schema diff, description length. Proxies. A definition under 25 characters gets flagged. A wrong definition of any length passes. Time elapsed and schema drift are the best stand-ins available to a tool that cannot check a definition against the answers it produces.
Our active_customer page trips none of these detectors. The source table updates daily. No column was renamed. The description is a healthy sentence long. By every proxy the catalog can measure, the definition is in perfect shape. It is also wrong by roughly the size of the trial funnel.
And the reverse holds too. A definition written four years ago can still be perfectly correct. Without a way to check it against reality, the catalog falls back on a calendar.
What fifteen vendors’ docs actually say
We reviewed the current documentation and release notes of fifteen catalog and governance products against four questions: who drafts definitions, what triggers maintenance, whether usage changes governed knowledge, and whether agents can execute queries. We did not run the products, so “not documented” is a statement about the public documentation, never about the product.
On writing definitions, the category has converged. An LLM drafts a description from technical metadata. A human accepts, edits, or discards it before anything is saved. Alation’s docs for AI-suggested descriptions state plainly that “the underlying data of a catalog object is never sent.” Databricks generates table comments from the schema and column names and requires review before saving. Snowflake and DataHub go further and let the model read sample values. The drafting side is getting solved, with sensible human gates.
Maintenance is where it thins out. Surely someone learns from the questions? Three vendors have real feedback loops, and the details matter.
Snowflake’s verified query suggestions are the strongest loop we found. Cortex Analyst analyzes recent query history available to the role and drafts up to ten additions to the semantic model at a time: verified queries, filters, metrics, descriptions, and synonyms. A human accepts, edits, or dismisses them.
Databricks Genie mines usage too. When an author gives a response a thumbs-up or downloads its results, Genie turns the query into a knowledge snippet. The snippets are capped at 200 per agent and stay in Genie’s own store rather than being written back to the catalog.
And Secoda redrafts on schema change. A new column in the source triggers an automation that regenerates descriptions, definitions, tags, owners, and PII status, with an optional review gate.
But look at what the three share. Snowflake suggests elements that were missing or frequently used. Genie’s thumbs-up and download signals reinforce what already worked. Secoda’s trigger is a schema diff, so a definition that became wrong for business reasons never fires the automation if the schema stayed put.
Across all fifteen, we found no vendor that documents detecting an incorrect definition and proposing the correction. The usage loops are additive: they learn from what worked or what is missing, not from a user saying, no, that answer is wrong.
The gap shows up in the vendors’ own writing. DataHub published a whole piece on why AI docs decay. It describes staleness detection for documentation in the conditional, as something a system should do. What ships today monitors whether the data table updated on time. Atlan’s best-practices page tells customers to capture production thumbs-downs and feed the failures back into the metadata themselves. Sound advice. Also a pipeline the customer has to build.
The question changes nothing
Back to our definition. Nothing about its age, schema, or description reveals the problem. In this case, its wrongness becomes observable only when someone uses it.
Someone asks how many active customers the company had last month. The answer comes back at 12,400. The person asking runs the revenue team and knows the paying base is around 9,000. Something is off. The trail leads to trials being counted, then to the March decision, then to the sentence in the catalog nobody updated.
Everything needed to fix the definition was present in that exchange: the question, the wrong number, the person who knew better, and the reason. And catalogs are not blind to questions anymore. They ingest query logs to rank popular assets. Several now have an assistant on top that answers questions from the metadata. Five of the fifteen expose SQL execution to agents through their own tool interfaces.
The question passes through the catalog, and nothing comes back. No drafted correction. No mark on the definition it used. The fix lands in Slack, the dashboard gets a filter, and the catalog page keeps saying what it said. Seeing the question was the easy part. Learning from it is the part nobody built.
A challenged answer is evidence, not yet a correction. The problem might be the business definition, but it might also be bad source data, a stale load, an incorrect join, an access rule, or an ambiguous question. The useful loop does not blindly rewrite the catalog. It gathers the question, answer, SQL, and user objection; identifies the likely cause; and routes a proposed change to the people accountable for that meaning.
This mattered less when people did the reading. Humans route around stale documentation; every data team quietly knows which pages to ignore. Agents do not. The ops lead at one SaaS scaleup put it to us bluntly: his team knows which dbt definitions are out of date, but an agent wandering through the warehouse has no way to know. His example was a table named for the paying customers it historically held and no longer holds. An agent that reads it to report ARR gets a number that is, in his words, completely wrong, and delivers it with full confidence.
The catalog stays current because it is part of the answer
If use is where semantic failures become observable, the fix is structural. Watching questions go by leaves the catalog unchanged. The loop has to close.
A governed definition has to be connected to the executable logic that produces the answer, with the SQL shown, rather than sit as prose a human may or may not consult. When a question exposes a gap, the system should bring back the evidence: what was asked, what definition and logic were used, why the result was challenged, which assets would be affected, and what change is proposed.
Nothing becomes truth automatically. An approved change needs an owner, an effective date, an audit trail, and a record of its downstream impact. The team can approve, edit, or reject it before the governed definition or its implementation changes.
Today, the steward chases the people who hold the knowledge and types in what they say. In this model, the questions do the chasing. The business owner decides what the term means. The steward evaluates the evidence and runs the approval. The technical owner applies and verifies the change. Same people, better work.
Would our March decision have survived this? The first time someone asked the active-customer question after trials launched, the number would have included trials. Someone who knew better would have said so. The correction would have arrived as a drafted edit to the definition, not as a Slack message that evaporates.
Usage closes an important loop; it does not replace every governance control. Low-use, high-impact definitions still deserve scheduled review. But for the definitions people and agents rely on every day, every challenged answer is a maintenance signal the catalog should not waste.
Discipline is a fine thing. But the same problem applies to anything an agent reads at face value: a semantic layer, a metrics file, or the docs in a repo. Each can be certified, owned, and quietly wrong. The first person to discover it is usually someone asking a question. Right now, the correction still ends up in Slack.
This is what we build at Cassis: the living data catalog. It stays current because it is part of the answer. Cassis uses the definitions your team governs to answer each question. When real use exposes a gap or stale definition, it comes back as a drafted update your team approves. If your catalog is certified, owned, and quietly wrong somewhere, we would love to compare notes: getcassis.com/data-catalog.
