CCAR-F · Study Guide

← Build Exercises

Domain 4

Prompt Engineering & Structured Output

6 build exercises to practice the concepts in this domain.

4.1Intermediate45 minutes

4.1 — Build an Explicit Criteria Code Review Prompt

What you will learn

  • Vague bars like "be conservative" or "high confidence only" do not hold in production.
  • Name the categories to flag and the categories to skip.
  • Calibrate severity with code examples, not adjectives.
  • Measure false positives, and turn off a category that keeps wasting reviewer trust.
  • Write the criteria first. Use confidence routing only after the criteria are explicit.
Open lesson 4.1
4.2Intermediate45 minutes

4.2 — Build a Few-Shot Enhanced Extraction Prompt

What you will learn

  • Reach for examples when formatting drifts, a judgement call is ambiguous, or a present field comes back empty.
  • Each example should include the reasoning, not only the input and the output.
  • Two to four examples, aimed at the failing cases, are enough.
  • Examples fix format and judgement. A schema change or a validation loop fixes a different failure.
  • Measure empty fields and format consistency before and after.
Open lesson 4.2
4.3Intermediate45 minutes

4.3 — Build a Structured Extraction Tool with JSON Schema

What you will learn

  • Optional or nullable fields stop the model from inventing a missing value.
  • tool_choice auto, any, and a forced tool name are three different guarantees.
  • A tool call removes syntax errors. It does not remove wrong values.
  • Unclear enums need an other value plus a detail string, and formats should be normalised in the schema.
Open lesson 4.3
4.4Advanced60 minutes

4.4 — Build a Validation-Retry Loop for Document Extraction

What you will learn

  • A retry sends the original document, the failed extraction, and the specific validation error.
  • Format, structure, and arithmetic can be fixed. A fact that was never in the document cannot.
  • A schema can carry calculated_total, stated_total, and conflict_detected.
  • Track the error pattern and which findings reviewers dismiss.
  • tool_use already removed syntax errors. The loop is for semantic checks.
Open lesson 4.4
4.5Intermediate45 minutes

4.5 — Design a Batch Processing Strategy

What you will learn

  • A blocking user-facing call stays synchronous. A latency-tolerant job can use the batch API.
  • custom_id ties each result back to its request.
  • Resubmit only the failed items, with a change aimed at the failure.
  • The processing window is up to 24 hours, so submission time has to fit the SLA.
  • Test the prompt on a sample before you submit the full batch.
Open lesson 4.5
4.6Advanced60 minutes

4.6 — Build a Multi-Pass Code Review System

What you will learn

  • A model that reviews its own output in the same session still holds the reasoning that produced it.
  • Review each file on its own, then run a separate pass over the findings.
  • Uneven depth, missed middle-file bugs, and contradictory flags are attention problems.
  • Route low-confidence findings to a person, using a threshold you checked on labelled examples.
  • A raw confidence number is not a calibrated threshold.
Open lesson 4.6