When to trust the answer

The one idea

AI models state wrong things with exactly the same confident tone as right things. This is called hallucination, and it is not a bug being fixed next month — it follows directly from how the models work (lesson 1: they predict plausible text).

So the question is never "is this tool trustworthy?" It is: "where did this answer come from?"

The two-bucket test

Bucket 1 — it transformed what I gave it. You pasted the report and asked for a summary. You gave it your notes and asked for an email. The facts came from you. Risk is low: read it over once like you would any draft.

Bucket 2 — it recalled something on its own. Names, dates, statistics, citations, laws, prices, medical or legal claims, "studies show". The model is reconstructing these from patterns, and it will sometimes produce a perfectly formatted citation to a paper that does not exist. Risk is high: verify against a real source before it leaves your desk.

Habits that make this automatic

• Ask the model to quote the exact passage it based each claim on (works when you supplied the document — if it cannot quote one, that claim is bucket 2).
• For bucket-2 facts, ask: "give me a source I can check" — then actually open the link.
• Numbers deserve special suspicion: totals, percentages, and dates are easy for a model to mangle even when summarizing your document, because arithmetic is not pattern-matching.

Your win today

Before you forward anything AI-assisted, run the two-bucket test on each factual claim: did I give it this, or did it recall this? Recalled claims get checked. That single habit removes most of the real-world risk of using these tools.