Stop predicting, start verifying

The original pitch for PermitCheck was seductive and wrong: upload your permit package and the tool tells you whether it'll be approved. Everyone I described it to nodded. It's the obvious product, and it's a promise you can't keep. I spent real time learning why before I admitted it.

The problem isn't that prediction is hard, it's that the thing being predicted doesn't live in the thing you're looking at. At the Committee of Adjustment, whether a variance is granted turns on the specifics of the request, on whether a neighbour shows up to object, on the panel and the day. None of that is in the applicant's drawings.

So no amount of training data makes the document predict the outcome, because the document doesn't contain the outcome's causes. I could've collected ten times the examples and the ceiling wouldn't have moved. The signal wasn't there to find.

The tell: when more data doesn't help

This is a useful diagnostic, and I wish I'd applied it sooner. If you're building a predictor and adding data isn't raising the ceiling, the question is probably wrong.

It usually means the target depends on variables your inputs don't contain. You're trying to predict the weather from a photo of the sky taken indoors. More photos won't help. You need a different question, one your inputs can answer.

For PermitCheck the answerable question was sitting right next to the unanswerable one. I couldn't tell you whether your package would be approved. I could tell you whether it was ready: is it complete, does the proposal comply with the zoning that governs this lot, where does it sit relative to the limits, what will need a variance. Every one of those is determined by the document and the property, so the tool can compute them and cite its sources.

Prediction and verification are different products

A predictor makes a claim you can't check until it's too late. A verifier makes a claim you can check the instant you get it. Here's what that looks like from a real decided case in my eval set: side setback, proposed 1.21m against a required 1.8m, Zoning By-law 569-2013 §10.20.40.70(3). You can look the clause up and check the claim yourself.

| | Predictor | Verifier | | -------------------------- | ------------------------------------ | -------------------------------------------------- | | The claim | Probabilistic, about the future | Checkable, about the present | | Checkable when it matters | No | Yes, the instant you receive it | | Errors | Invisible until it's too late | Auditable per output | | Trust rests on | A track record taken on faith | Each answer citing the rule and the value it read | | Abstaining | Not available | Says so when it's unsure |

That difference is what made the rest of the work possible. Once the promise was verification, I could measure it honestly, because each answer had a ground truth I could check against. I could enforce that the tool never claims confidence it can't support, and I could let it abstain. None of those moves are available to a predictor.

Where else this applies

A lot of AI products predict something they should instead verify, or verify but overreach into prediction because prediction demos better. Before you commit, separate the two. Ask what your inputs actually determine, not what you wish they determined. If the valuable-sounding question depends on facts your data doesn't hold, no model improvement will get you there, and the keepable version is often standing right beside it.

I stopped trying to predict the verdict and started verifying the package.