The Most Useful Thing a Tool Can Say Is "I Don't Know"

I build small tools because I like the way my head feels when I’m working a problem. One of them is file-observer: you point it at a folder, it walks every file, and it writes down what each one is — type, size, whatever metadata it can read, a checksum. One read-only pass, one boring report. It looks, it describes, and it changes nothing.

“Changes nothing” sounds like the half I didn’t finish. It’s the half that matters. Every honest line in that report is there because, at some point, guessing would have been easier and would have looked better, and I said no. These are the refusals that taught me the most.

It refuses to reconcile

A video file can carry two different “created” timestamps. One (the mvhd creation date) is really when the file was last finalized, in UTC. The other (Apple’s QuickTime creation date) is the actual moment of the shot, in the camera’s timezone. Most of the time they agree. Sometimes they’re hours apart.

The helpful move is obvious: pick one, or split the difference, and hand back a single clean “created” field. The tool refuses. It reports both, raw, and never reconciles them.

Here’s what made me sure. I had a clip that had been AirDropped from one phone to another. The transfer rewrote the file, so the “finalized” timestamp moved — but the “moment of the shot” didn’t. The two fields disagreed by a wide margin, and that disagreement was the story: it was the evidence the file had been moved after it was recorded. If I’d collapsed the two into one tidy number, I’d have deleted the one signal worth having, and picked at random between “when it was shot” and “when it was last touched.” The gap isn’t noise to clean up. The gap is the point.

It refuses to guess

Ask it how many pages a PDF has and, for a long stretch, on a lot of files, it would tell you: nothing. Just null.

That is an embarrassing thing to ship. “How many pages” is about the most basic question you can ask a document, and a tool that shrugs at it looks broken. But modern PDFs often tuck the page count into a compressed structure the quick read never reaches, so the honest answer really was “I didn’t see it.”

The tempting fix was to grab the biggest page-ish number anywhere in the file and call that the count. I tried it. On a short spec it confidently reported several times the real page count, because it had counted the entries in the bookmark outline instead of the actual pages. A guess that looks authoritative and is wrong is worse than no answer, because now you trust it. So the rule became a single line: null never means the answer is wrong. It means we didn’t observe this within our bounds. “Not seen” is not “not there,” and pretending otherwise just to look complete is the one thing the tool won’t do.

Later I did the real work to actually read those compressed structures — but only behind a hard gate: it had to agree exactly with a separate reference parser across a whole corpus before it was allowed to answer at all. Earn the number honestly, or keep saying “I don’t know.”

The day “deterministic” wasn’t

The property I’m quietly proud of is the boring one: same bytes in, same report out, byte for byte, sealed with a checksum. You’d think that’s the easy part. It is not. “Deterministic” turns out to be a property you keep discovering you don’t have, in new and creative ways.

The first leak was the wall clock. A file got deleted mid-scan, and the code stamped the current time into the record where the file’s own timestamp belonged — and that value fed the checksum. Same folder, scanned twice, two different checksums. The catch is the part worth telling: my own tests asked “did it crash? did it stay within bounds?” and passed it clean. It took a different reviewer — a different AI model, deliberately prompted to think unlike the first one — to ask the question mine never did: “is this actually deterministic?” (That habit of hiring a reviewer chosen to fail differently than you is its own whole story.)

There were more. A dependency’s version got rendered as a raw object and leaked a memory address — a per-process pointer — straight into the “reproducible” output. An operating-system-dependent file sort quietly reordered the list on one filesystem. Every one of them taught the same lesson: anything that feeds the checksum has to come from the file’s own bytes and a declared environment, and nothing else — not the wall clock, not a pointer, not the operating system’s mood. Determinism isn’t something you assert once. It’s the property you keep failing to have until you’ve named every channel the messy world leaks in through.

Where refusing costs you

I’d be selling you something if I stopped there, so here’s the bill.

Point it at a scanned paper document and it says, in effect, “no text here, you’ll need a vision model.” The user wanted the words. They got a diagnosis. For them, a worse tool that just ran OCR and handed over a messy best guess would have been more useful than my honest one. And that page-count null I’m so proud of — for a couple of release cycles it meant most of the PDFs in a big pile got no page count at all. A real person who just wanted a rough sense of how long a document was got nothing, from a tool named for observing files. There are even a few specific shapes of real conversation it won’t recognize — an interview laid out as a list of Q: and A:, say — because the test that would catch them also fires on FAQ pages and data tables, so I left the bar where it was and wrote the gap down instead of lowering it.

That’s the trade, stated plainly. Honest nulls push work downstream. Refusing to reconcile means you do the reconciling. The tool observes; it never renders a verdict — it will tell you a file has macros, never that it’s malicious. If what you want is a tool that hands you the answer, this is the wrong one, on purpose.

Why the refusals count as discipline

Here’s what I didn’t understand starting out. From the outside, “correct or null, and the null is a promise to go earn the real value later” and “null because we couldn’t be bothered” look exactly the same. Same empty field. The only thing that tells them apart is showing the cost and the payoff in the open, wince included. The refusals only read as discipline because the ledger is public.

And it matters most at the very front of a pipeline. The first stage is the one every later stage inherits from. If it guesses, everything downstream builds on that guess without ever knowing it’s a guess — the mistake compounds quietly, stage after stage, and nobody can see where it entered.

flowchart TB
  bytes["the raw files"] --> obs["stage one: observe, change nothing"]
  obs --> manifest["one honest report, correct or null"]
  manifest --> a["a search index"]
  manifest --> b["a routing rule"]
  manifest --> c["whatever comes next"]

Which is the whole reason I built a tool whose proudest feature is a shrug. The most valuable thing that first stage can ever say — the thing worth engineering it to be able to say honestly — is I don’t know.

I’m an enthusiast, not a professional. I can go months without talking to anyone who writes code for a living, so if you do this for real and you can see where this discipline breaks — or where I’ve quietly talked myself into calling a limitation a virtue — I want to hear it. The tool’s out in the open. Come poke holes.