Evaluation Checklist
- Did it answer what was asked?
- Is anything material missing (caveats, limits, deadlines)?
- Are specifics supportable (numbers, names, quotes, citations)?
- Is it consistent within itself and across runs?
- Is it balanced, or one-sided?
- Does it need human review before use?
Build your checklist from the source before you read the output, so omissions are visible.
Hallucination Red Flags
| Red flag | What to do |
|---|---|
| Precise statistics with no source | Verify against the source of record |
| Citations you cannot locate | Treat as unverified until found |
| Detail beyond what the document says | Check against the document and remove unsupported detail |
| Same confident tone for every claim | Tone is not evidence. Validate the claims that matter |
| Different figures on a rerun | Investigate. Disagreement shows where the risk is |
Decision Table
| If the question says... | The answer is likely... |
|---|---|
| Figure will drive a decision | Verify against the source of record; add independent confirmation |
| Claude says it is confident | Not validation. Check the evidence |
| Summary is accurate but leaves out a caveat | Failure: omission of material caveats counts |
| Output goes to customers, regulators, or press | Human review before it is used |
| Decision affects a person's job, pay, health, or legal status | A qualified person decides. Claude only informs |
| Same facts for different readers | Adapt tone and length, then compare versions to confirm the facts match |
| Reusable or iterated standalone content | Artifact |
| Comparing several options across attributes | Table |
| Short conversational answer | Inline reply |
Traps
| Trap | Correct Answer |
|---|---|
| Ask Claude "are you sure?" to validate | Verify against an independent source of record |
| Fluent output means accurate output | Fluency says nothing about accuracy |
| Every sentence is true, so it is complete | Check what is missing as well as what is stated |
| Rewrite for audience changes the facts | Audience changes tone and detail; facts stay identical |
| Use an Artifact for every answer | Use one for standalone, reusable, or iterated content |
If You Remember Nothing Else
- Verify consequential specifics against the source of record.
- Self-reported confidence is not validation.
- Omission of material caveats counts as failure.
- External-facing, legal, financial, health, and people decisions need human review.
- Domain 2 is the heaviest at 21%. Spend time here.