The chart is beautiful. That is the problem. AI tools now produce analysis in the finished form managers are trained to trust — clean tables, confident trend lines, a summary that reads like a consultant wrote it — at a speed that leaves no pause for doubt. The analysis may be right; these tools are genuinely capable at arithmetic, pattern-spotting, and first drafts of interpretation. But the failure modes have moved. The old spreadsheet’s errors were usually visible: the broken formula, the wrong range, the typo a reviewer could hunt. A machine-generated analysis fails more politely. It averages things that should never share a denominator, compares periods of different lengths as if they were the same, drops the awkward subgroup into the total, and narrates all of it in prose so fluent that the numbers inherit credibility they never earned. The manager’s job has changed accordingly: less building analysis, more auditing it — and auditing has its own discipline, older than the tools and badly in need of dusting off.
Four questions that catch most of it
The checking discipline compresses, in practice, to four questions asked in order. First: what does the metric actually count? Not its label — its definition, its source field, its exclusions. A “customer” that counts trials this quarter and contracts last quarter manufactures a trend out of bookkeeping. Second: what is the denominator? Rates without visible bases are how a change from two cases to three becomes a dramatic surge; every percentage in the deck should be able to produce its whole number on demand. Third: what would this look like disaggregated? Totals are where contradictory stories go to hide — a flat company line can be one region climbing and another collapsing — and a tool optimizing for a coherent narrative will rarely split the data against its own summary unless asked. Fourth, the base-rate question: against what should this number be compared — last period, the plan, the same season last year — and who chose that comparison? The baseline is the argument; everything else is arithmetic. None of these questions requires statistical training. They require the posture of a reviewer rather than an audience.
Spot-check like an auditor, not a fan
Beyond the four questions sits a sampling habit borrowed from audit work: pick a small number of the analysis’s load-bearing figures and recompute them from the source data by hand — not all of them, which nobody has time for, but enough, chosen unpredictably, that the tool’s overall reliability becomes an observed fact rather than an assumed one. When the spot checks pass repeatedly, trust is earned and review can lighten, for that tool, on that data. When one fails, the whole output is reclassified as a draft, and the failure — a mislabeled field, a silent filter, a date range that crept — gets written down where the team can learn it, because machine errors are systematic: the figure that is wrong today for that reason will be wrong again next month for the same one. One worked example clarifies the scale of the stakes: an analysis reporting average resolution time improved (illustrative numbers, not research) can be counting only the tickets the new system closed, having silently excluded the backlog it inherited — every figure accurate, the conclusion false. The chart lied politely. The questions are how you decline to be lied to.
Teach the posture downward, because the analysis habit now forms early in careers that once spent years building the numbers by hand and learning where bodies get buried. A junior analyst who has only ever reviewed machine output may never develop the instinct that a total looks wrong — the felt sense, earned from manual work, that this figure cannot be right because the operation it describes could not have produced it. Managers can rebuild that instinct deliberately: walk the team through one real analysis a month, errors and all, asking the four questions aloud until asking them becomes the culture. What is being protected is not arithmetic. It is the organization’s ability to tell the difference between a measurement and a story — a difference the tools, left unaudited, will blur a little more fluently every quarter.
Experience it, don’t just read about it
Data-Driven Decision Making — Signature Course No. 49 — builds the full discipline across 18 concepts, coached by conversation and tested against decisions from your own work. Its two capstones are the work itself: The Decision Framed Right and The Recommendation That Holds Up. It is $195; one Corporate Library License covers it for every enrolled employee on a team. Want to see the coaching first? The free sample of Breakthrough to Influence runs the same Lighthouse Coach method.
© 2026 Cognosco Learning. All rights reserved.