Totally Meaningless, Totally Impressive

How a machine wrote a 24,000-page data dictionary overnight(ish) — and why the page count is the least interesting thing about it.

---

A while ago I ran a job and then, for fun, pretty-formatted its output and counted the lines. A little over 1.2 million. Divide by fifty lines to a page and you get 24,396 pages.

Twenty-four thousand pages of data documentation. Generated deterministically. No human author.

That number is completely meaningless. It’s whitespace and brace-counting; I could double it by changing the indent from two spaces to four. It is also the kind of stat that makes a room go quiet. For a second. Until someone asks “is any of it real?“

The graveyard problem

Every large company that runs on data has the same dirty secret: its data catalog is a graveyard. Hundreds of thousands of columns across thousands of tables, and the “description” field is empty on almost all of them. The ones that aren’t empty can be worse — a stale sentence someone wrote in 2019 for a column that’s since changed meaning twice.

Nobody fills them in, because filling them in is nobody’s job (it’s mine, actually), and because the person who could write a good description for locationAsnWithOwner is the one person too busy to do it (you know who you are). The catalog becomes a place where documentation goes to not exist.

Here’s an idea! Let’s point a large language model at the problem! “DESCRIBE THIS COLUMN!” Problem solved.

It sort of works, and it’s sort of a disaster — because an LLM will confidently describe every column: the ones holding raw device identifiers, the cryptic double called avg_lpeel_v2 it should never summarize, the ones that are 99% null, and the ones a human already documented perfectly well. You don’t have a catalog anymore. You have a chatty intern who never says “I don’t know” and never defers to the expert in the room (you know who you are).

(Don’t believe me? Ask the internet what lpeel means. It told me, with total confidence, that it’s a peelable packaging film. Or a Korean chemical peel. Or a paragraph-writing framework. Or a stock ticker. It is none of these.)

Measure, then decide — but never in the same place

The system Claude & I vibe-coded takes a different shape, and the shape is the whole idea.

It splits into two kinds of thing: producers and a resolver.

Producers are specialists. One reads the physical layout of a table and knows which columns are partitions. One maintains a canonical dictionary of “primitive” concepts — the handful of identifiers and enums that recur across hundreds of tables, each defined once, by a human (me), authoritatively. One profiles the actual values in text columns and writes a plain-language paraphrase. One scans for anything that looks like personal or regulated data.

Here is the discipline that makes it work: producers only measure and flag. They never decide, and they never throw anything away. A producer that finds a column is 99% null writes down “99% null” and moves on. A producer that suspects a column is a personal identifier flags it and withholds its own guess — but records that it withheld, and why. Nothing low-quality is discarded; it’s retained, scored, and labeled. The producers are librarians, not editors.

And each producer can feed another: a sparse-column producer can run big table scans on the small subset of columns previously marked as 99% null. A locally run small-language-model producer can describe PII columns without exposing the values to Anthropic or OpenAI.

But all the deciding happens in exactly one place: the resolver.

One answer per column, and a receipt

The resolver’s job is to look at every candidate description for a column — the human dictionary definition, the native comment someone typed into the database years ago, the partition template, the AI paraphrase — and pick one winner, by a fixed order of precedence:

1. A curated canonical definition always wins. The whole point of defining account_id once is that it beats everything else, everywhere.
2. Then a human-authored comment. The machine loses to a person.
3. Then the deterministic partition description.
4. Then, and only then, the AI paraphrase. The model is the fallback, not the default.

Crucially, the resolver runs no model at all. It’s precedence and policy — pure, deterministic, boring, a query I can change over and over because all the intelligence has already been persisted, upstream, by the producers. The resolver is the referee, and referees shouldn’t improvise.

And every answer ships with breadcrumbs. Each resolved description carries its provenance: which source won, when it was generated, and — this is the part people underestimate — a publication decision. Every column is marked publish, suppressed, or omitted, each with a reason. A column the system deliberately declined to describe (because it’s a raw identifier, say) doesn’t silently vanish. It ships a blank description and the reason the blank is there. “I looked at this and chose not to describe it” is a first-class, auditable output — the exact thing the fluent intern can’t give you.

Because nothing is thrown away and every line is drawn in one place, moving a line is cheap. Want to publish lower-confidence descriptions next quarter? That’s a one-line change to a policy, re-run in minutes — not a re-measurement of a hundred thousand columns. The arbitrary decisions are reversible because the evidence underneath them is permanent.

I built it, but I don’t like it

I built a fleet of agentic tools to fill a web-based data catalog like Alation. I did it because that’s what the organization wants. They don’t want the context harness I’ve talked about in previous posts that has me doing ten points of pre-AI engineering work in an hour.

And that has given me a new perspective on AI. We’re all using it to automate the difficult part of our job, but we’re not changing our expectations of how our users will leverage AI. Governance wants to use AI to fill the catalog because they’ve been trying to get us PMs to fill it for years. I use AI to find bugs and outliers in the data because that’s been the hard, time-slog of my job for years. My engineers use AI to manage Jira and run boring tests because that’s the part of the job they don’t want to spend time on.

My gut: headless is the future. I’ve built a fleet of agentic tools that will fill a web UX that no-one will use in a year with descriptions too-high-level to be valuable.

But my gut instinct is based solely on the fact that I don’t want that. So who knows?

Oh — is any of it real? Mostly: the first run that spanned real calendar time went twelve days between passes, and a couple dozen tables got rebuilt or dropped underneath it, so a fraction of a percent of those pages now faithfully document ghosts. I know exactly which ones, because every answer carries a receipt — which was the entire point.

A party trick

The page count is a party trick. It’s going on a slide because a room that goes quiet is a room that’s listening. And if the room erupts instead, it’s erupting because I have their attention. They’re hooked. 24,396 is a very good number for getting a reaction.