Does AI Check Its Work? The Hubble Mirror Test

Only when it has been built to, and only when the people relying on it insist. Most current systems can fetch a document, recompute a figure, or report that a number is missing. Whether they do is a habit rather than a capability, and it varies sharply between tools that look equally impressive in a demonstration. Access to a check is not the same as the check being run.

The Hubble Space Telescope is the cleanest illustration of what that gap costs. Its primary mirror was ground over years, at enormous expense, to extraordinary precision — and to the wrong shape. The contractor polished it against one elaborate custom instrument, a reflective null corrector, in which a lens had been set roughly 1.3 millimetres out of position.¹ The mirror faithfully followed the bad ruler and came out too flat at the outer edge by about 2.2 microns,² a fraction of the thickness of a sheet of paper and catastrophic at that scale. Two simpler instruments indicated an error during fabrication, and both readings were discounted as flaws in the simpler tests⁴ (NASA's account of the mirror flaw). The telescope launched in 1990 unable to focus perfectly³; astronauts installed corrective optics in 1993.¹ The checks existed. They were not believed.

Owners weighing an analytical tool — or an advisor, a banker, a management team — face the same choice. The questions below test two things: whether a check was run at all, and whether a disagreement was allowed to matter. Most of them are not really about software.

When a figure isn't in the filing, does it say so?

A good answer is a flat sentence: that number does not appear in this document. It may add where such a figure would normally sit, and then it stops. Reporting an absence is exactly what the two simpler Hubble instruments did — unglamorous, and correct.

A weak answer is fluent. It produces a plausible figure in the right units, wrapped in a confident sentence and attached to no source line, or offers an estimate without labelling it a guess. Being wrong is recoverable. Being fluently wrong survives review.

This property outranks everything else on the list. A system that fills gaps cannot be trusted with a filing however capable it is in general — and the same standard separates a useful broker from a dangerous one. If the number is not there, the only acceptable answer is that it is not there.

Can it point to the line it read, or is it recalling the company?

A good answer cites: the page, the note, the table. More importantly, it separates what it has just read from what it already knew — this figure comes from the statement in hand; separately, the company has historically described this segment in these terms. Two sources, clearly labelled.

A weak answer sounds informed. Because it "knows" the company, a summary of one document fills up with facts that were never in that document. Some of those recollections are correct. They are still the wrong kind of correct, because nobody can tell which sentence was actually read. Which document to believe is a later question; whether there is a document at all is the first one.

The same request works on a human memo: where exactly did this come from? The answer should be a page, not a recollection.

Given a way to fetch the real document, does it actually fetch it?

Every new release now leads with tool access — the ability to pull a live filing or run a calculation rather than answer from memory. Access is not use. Every system has a knowledge cutoff, and companies keep filing.

A good answer shows the fetch. It names the document retrieved and the date on it, or it says plainly that the question needs current paper and stops. Staleness said out loud is a form of honesty. Whether these tools change how a filing gets read is a separate subject; it does not settle whether the tool was opened this time.

A weak answer has the tool, never calls it, and writes in the present tense about an old world — because recalling is faster than fetching, and sounding current is rewarded. The human version is familiar: an advisor whose view of a business was formed years ago and has not been disturbed since.

Does it keep millions and billions straight?

Units are where confident analysis dies quietly. Millions read as billions. Basis points — hundredths of a percentage point — read as percent. A fiscal year mistaken for a calendar year. A restated figure set beside the number as first reported. The arithmetic is trivial; the discipline is not.

A good answer treats the unit as part of the fact, states it beside the figure, and flags a conversion instead of folding it into a sentence. A weak one wraps polished prose around a number off by a factor of a thousand, and because the prose is polished, nobody rereads it.

Anyone who has run a company knows this failure from their own finance team. The slide was beautiful and the decimal was in the wrong place. Confident presentation is not evidence of a checked number; it is what keeps an unchecked one alive.

Told what the reader already believes, does it still disagree?

This is the test most people never run, because the result is pleasant. State a view — these margins look unsustainable — and ask for analysis. A good answer tests the view against the document, is willing to say the evidence points the other way, and can name what would have to be true for the stated view to hold.

A weak answer reflects the conviction straight back, dressed in supporting detail: it finds the helpful lines, softens the footnote that cuts against, and calls the result balanced. The remedy is simple — ask the same question twice, leaning each way, and see whether the answers follow the lean.

Agreement as a service is disqualifying in research, because it manufactures false confirmation on a decision that already had conviction. It is David Swensen's test applied to a machine: an analysis that cannot contradict the person paying for it has already picked a side.

Asked the same question tomorrow, does the answer hold?

A good tool is dull here on purpose. Same document, same question, asked again next week: the figure, the source and the refusal to guess all match. Wording may move. The number may not.

A weak one writes a fresh essay each time — one pass cites the note, the next recalls a headline, the third agrees with a hint buried in the question. Variance that would be charming in conversation makes a repeatable process impossible, and an analysis that cannot be reproduced cannot be audited.

This is also the honest way to test a large "context window" — the amount of text a system can hold at once, and a marketing number on its own. Ask for a specific disclosure buried in the middle of a long report, twice. The middle is where the interesting disclosures live.

What does the hundredth document cost, not the first?

Reading one annual report well is a demonstration. The work around a real decision is several years of filings, the competitors, the footnotes — or thousands of earnings calls read the same way each time. At that scale, cost and speed stop being logistics and become quality questions.

A good answer prices the job plainly: what it costs across the pile, where it slows, where a person is still required. A weak one is dazzling on one famous company and silent about the hundredth document.

Speed bought by quietly dropping the checks above is not a saving. It is schedule pressure — the same force that waved away two instruments on a factory floor because stopping was expensive.

Use the list as a filter, not a scorecard. One failure — a fabricated figure, a units slip, a memory dressed as a citation, agreement on demand — is enough to set an analysis aside until it is repaired. The questions work the same way on software in a demonstration and on people being paid a retainer. The elaborate instrument at Hubble was not trusted because it had earned belief; it was trusted because nothing simpler was allowed to stop the work.