AI Drafts, You Decide: Checking an AI-Prepared Recommendation

AI Drafts, You Decide: Checking an AI-Prepared Recommendation

A recommendation lands in your inbox. It’s well structured, it cites figures, it weighs the options and it reaches a clear conclusion. Some or all of it was drafted with AI. It reads like something you could approve this afternoon.

The question is whether anyone has checked it. Fluent writing and checked evidence look the same on the page. For a decision that’s cheap to undo, that might not matter much. For one that’s costly to reverse, it matters a lot.

What helps is a short decision memo that shows, in one place, the options, the evidence and where it came from, who checked what and what they found, what’s still unresolved, the strongest argument against, who owns each risk, and who is making the call. The template is at the end of this article. You can start filling it in straight away, though checking the evidence takes as long as it takes.

AI drafts, a person decides

AI can help prepare a decision. It can’t be accountable for one. A named person makes the call, and that person is responsible for it whether or not a model wrote the first draft.

This approach needs three things to work with:

If the evidence for a claim doesn’t exist or can’t be opened, the claim isn’t verified. Write it down as unresolved. That’s a legitimate place for a memo to be, and much better than a draft that hides the gap.

Where it helps

These are uses I’d suggest trying. They aren’t promises that the output will be accurate or the decision better.

In each case the output is a starting point for a person to check, edit and own.

Where it misleads

The risk isn’t that the output looks wrong. It’s that it looks finished. Three distinctions matter.

Plausible prose isn’t verified evidence. A generated summary, figure or citation can be fluent and wrong. A reference can exist and still not say what the summary claims, or say it about a different population, period or task. Until someone has opened the source, the claim is unchecked.

A scenario isn’t a forecast. Asking a model what might happen if a competitor cuts prices gives you a scenario: a story worth testing. A forecast you can rely on needs historical data, validation against outcomes the method didn’t see, and a stated error rate. If the recommendation has none of those, call it a scenario and test it like one.

A list of failure modes isn’t a validated simulation. A model can suggest ways a plan might go wrong, and that’s useful for widening what you consider. It hasn’t modelled your organisation or tested anything. Treat each suggested failure mode as a prompt for a person to assess, and give every material risk a named owner: someone responsible for watching it and acting if it shows up.

Checking sources

Checking means going to the underlying source, not re-reading the summary or asking the model whether it’s sure. For each claim the recommendation depends on:

  1. Open the source. The actual report, dataset, contract or page, not a generated description of it.
  2. Find the passage. Locate where it supports the claim. If you can’t find it, the claim is unsupported.
  3. Check the scope. Note the source’s date, what task or question it covered, which people or customers it described, and any limitations it states. A real source can still be the wrong evidence for this decision.
  4. Record the check. Write down who checked it, what they checked and the result: supported, narrower than claimed, unsupported or contradicted.

When a claim is unsupported or contradicted, it stays visible as unresolved. Get a person with relevant knowledge to review it, and reconsider whether the recommendation still stands before relying on it. Recording a source in the memo isn’t the same as checking it; the memo keeps those as separate entries for that reason.

Illustrative example. A draft recommendation to drop a product tier says churn rose after the last price change, citing a quarterly retention report. The checker opens the report and finds the churn figure, but it covers only self-serve customers, and the report notes the quarter included a billing outage. The memo records the source, the check, and the result: narrower than claimed, with a confounding event. The claim moves to the unresolved list until someone looks at the other customer segments.

As of 2024: I asked ChatGPT to help draft some technical content about encryption and cryptography. It told me, with complete confidence, that something I’d understood to be impossible could now be done. It was convincing enough that I spent hours reading blogs and research papers to catch up, only to find there was nothing to catch up on. My original understanding was right and the answer was wrong.

That’s one experience with one tool in 2024. It says nothing about how often current tools get things wrong. The point is about checking: a confident answer shook my understanding of a subject I was supposed to know. On a subject I didn’t know, I’d have had little chance of spotting it without going to the sources.

The decision memo

Copy this into whatever you write documents in. Fill in what you know, and leave gaps visible rather than smoothing them over.

DECISION MEMO

Decision:
  What is being decided, and by when?

Options:
  1.
  2.
  (Include doing nothing or waiting, if it's genuinely an option.)

Evidence (sources recorded, not yet checked):
  - Claim:
    Source:
    Source date:
    Scope and stated limitations:

Checks completed:
  - Claim:
    Checked by:
    What they checked:
    Result: supported / narrower than claimed / unsupported / contradicted

Unresolved assumptions and claims:
  -

Strongest counter-argument:
  

Material risks and owners:
  - Risk:
    Owner (named person):

Decider (named person):

Decision and rationale:
  

Revisit date:

A claim belongs under Checks completed only once a person has opened its source and recorded the result. Anything still unchecked, unsupported or contradicted goes under Unresolved so the decider sees it before deciding. The revisit date is when you’ll look again at whether the assumptions held.

If you’re thinking about how decisions get prepared across a whole week rather than one at a time, I’ve written about what decision readiness means for executives.

Use this memo for your next decision that would be costly to reverse. Let AI draft whatever it can; then check the sources, keep the unresolved items in view, and make sure a named person owns the call.

Glenn Gillen is a product leader, early-stage investor, and writer.

Want help turning ideas like these into reality? Let's talk.