Domain 5: Governance, Safety & Risk Management

Governance terms: guardrails, failure modes, HITL, compliance regimes, and ethical AI controls.

• Deterministic guardrail

  A code-path, hook, or policy-engine check that blocks or allows an action independently of model wording — required when requirements say must always / must never.

  Exam Prompts alone do not satisfy hard safety language.

  See also: 5.1 Guardrails & safety controls

• Layered safety controls

  Combining prompt guidance, input/output filters, tool allow/deny lists, and escalation so no single soft control is the only barrier.

  Exam Prefer defense in depth when the model still attempts forbidden actions.

  See also: 5.1 Guardrails & safety controls

• Failure-mode register

  An enumerated list of LLM failure modes (hallucination, tool misuse, outages, drift, policy gaps) with user impact, detection, mitigations, and residual risk.

  Exam Optimistic “the model will handle it” answers fail.

  See also: 5.2 Risks & failure modes

• Residual risk

  Risk that remains after mitigations; must be documented and accepted by named owners on a review cadence — not hidden or assigned only to a vendor.

  Exam Residual risk without ownership is incomplete.

  See also: 5.2 Risks & failure modes

• Structured HITL handoff

  A decision packet for human reviewers: case facts, model suggestion, uncertainty/evidence, and the required decision — not emoji pings or raw dumps alone.

  Exam HITL without structured handoff is incomplete.

  See also: 5.3 Human-in-the-loop

• Escalation trigger

  A measurable condition that routes work to humans (risk class, policy gap, user request, agent stuck). Uncalibrated confidence alone is a weak sole gate.

  Exam Prefer measurable triggers over confidence-only escalation.

  See also: 5.3 Human-in-the-loop

• Regime mapping (GDPR / HIPAA / FedRAMP)

  Determining which regulatory obligations apply from data classes, regions, and deployment context, then designing controls and evidence for that regime.

  Exam Generic best-practice slogans without mapping to the named regime are weak.

  See also: 5.4 Regulatory compliance

• Data minimization in Claude paths

  Stripping or redacting unnecessary sensitive data from prompts, retrieval, logs, traces, and eval stores before it spreads.

  Exam Full PHI/PII prompt logging “for debug” is a compliance red flag.

  See also: 5.4 Regulatory compliance

• Disparate outcome testing

  Measuring error or recommendation rates across relevant groups/segments to detect unfair patterns that overall averages hide.

  Exam Ethics items reward slice measurement over slogans.

  See also: 5.5 Ethical AI considerations

• Transparency & recourse

  Disclosing AI involvement and material limitations when required, and giving affected users appeal, human review, or correction paths.

  Exam Hiding the AI or providing no recourse fails fairness/transparency stems.

  See also: 5.5 Ethical AI considerations