Debugging and Error Handling

What this skill covers

Domain 4 is Eval, Testing, and Debugging, and it is 2.6% of the CCDV-F exam. The task in this lesson is debugging and error handling: identify the error type, choose a recovery, read the trace, and isolate an integration-layer problem from a model-output problem.

The exam is asking where you look first. A prompt rewrite does not fix a 401. A retry does not finish a tool call you never returned. A schema failure on a complete end_turn response is not an API outage.

Two different signals

An error means the Messages request was rejected or failed while the service handled it. The body is JSON with type error, an error object that has type and message, and a request_id. There is no assistant answer to parse.

stop_reason is on a successful response. It says why generation stopped. end_turn means Claude finished. max_tokens means your output cap was hit. stop_sequence means one of your stop sequences appeared. tool_use means you must run the client tools and send the results. pause_turn means a server-tool loop hit its iteration limit and you continue by sending that assistant message back. refusal means Claude declined, and stop_details says more. model_context_window_exceeded means the response filled the model window and the text is truncated.

Streaming can return HTTP 200 and then an error event later in the stream. That event is still an error. The partial tokens before it are not a finished answer.

Error identification

Read the HTTP status and error.type together. Catch the SDK's typed exception. Do not branch on the human message string. The type can gain new values over time. The request-id header, and request_id in the error body, is what you keep for support. On Claude Platform on AWS, keep the AWS request id for CloudTrail and the Anthropic request id for Anthropic.

A 400 invalid_request_error is the request shape: missing model, max_tokens, or messages, roles that do not alternate, a tool_use id with no tool_result immediately after, a tool_result that is not the first content in that user message, or a thinking block that was edited or dropped before you sent the assistant turn back. A bad model id is a 404 not_found_error. A malformed or revoked key is 401 authentication_error. A key that cannot use the resource is 403 permission_error. A billing problem is 402 billing_error. A body over the size limit is 413 request_too_large. A conflict with current state is 409 conflict_error.

429 rate_limit_error can be a rate limit, a usage-tier spend cap, a workspace spend limit, or a short acceleration limit. A tier spend-cap 429 has no retry-after header and keeps failing until access resumes. 500 api_error, 529 overloaded_error, and connection failures are service-side. 504 timeout_error means the request ran long. Long generations belong on the streaming Messages API.

• 400, 401, 402, 403, 404, 413: fix the request, the key, billing, or the payload. A blind retry repeats the same failure.
• 409: resolve the conflict, then retry.
• 429 with retry-after: back off. 429 with no retry-after on a spend cap: wait until the cap lifts.
• 500, 529, timeouts, and dropped connections: retry with backoff. The official SDKs already do that twice by default.

Recovery strategies

Recovery is the next action that matches the signal you already identified. Retry is for transient failures. The official SDKs retry connection errors, rate limits, and 5xx responses with exponential backoff, twice by default, and they honor retry-after when it is present. A second retry loop around the SDK retries the same call again. Turn the SDK retries up or down on the client when you need a different budget. Do not also string-match the message.

A tool_use stop is not a failure and not a retry. Execute every client tool, then append the assistant message exactly as you received it and a user message whose tool_result blocks come first, one result per tool_use id. If the tool itself failed, set is_error to true and put an actionable reason in the result. Claude can choose another step. An empty string with is_error false tells the model the tool succeeded and found nothing.

pause_turn is the other continuation. Send the assistant content back unchanged so the server-tool loop can proceed. Do not invent client tool_result blocks for that stop. A response that is waiting on your tools uses tool_use, not pause_turn.

max_tokens means the text you have is a prefix. Raise the cap or continue from that assistant message. Do not validate the prefix as if it were a finished object. model_context_window_exceeded is also truncation, and a larger max_tokens does not create window space the input already used. Shorten the history or the tool output, then call again. refusal is a decline: read stop_details and send the task to a fallback model when the product allows it. stop_sequence is the stop you configured. Read which sequence fired before you call it a bug.

if (response.stop_reason === "tool_use") {
  const results = await runClientTools(response.content);
  messages.push({ role: "assistant", content: response.content });
  messages.push({ role: "user", content: results });
} else if (response.stop_reason === "pause_turn") {
  messages.push({ role: "assistant", content: response.content });
} else if (response.stop_reason === "max_tokens") {
  // prefix only: raise the cap or continue, then parse
}

Trace analysis

A trace is one turn and the next request that used it. For a rejected call, store the status, error.type, the message, and request_id. For a 200, store the model, stop_reason, stop_sequence, output token count against max_tokens, the content block types, and each tool_use id paired with the tool_result you sent, including is_error. That row is enough to tell a truncation from a bad answer and a missing tool result from a model mistake.

Walk the row in order. No 200 means you never received model output. tool_use or pause_turn means the harness stopped early. output tokens equal to max_tokens, or stop_reason max_tokens, means the text is cut off. end_turn with output tokens under the cap means generation finished, and the remaining question is whether the content passes your check.

Failure modes you should be able to name from that row: the request was rejected, the stream died after 200, the output was truncated, the tool loop was not closed, the tool failed and the app hid it, the model finished and the content is wrong, or the same prompt varied because sampling is not a contract. Non-determinism is not an HTTP error. Pin the behavior with checks, not with a retry that you treat as success when any sample passes.

Integration layer versus model output

The integration layer is everything your process does around the model: the key, the URL, the JSON you assemble, the tool runner, the parser, and the retry policy. Model output is the assistant content on a request the API accepted and finished.

Use the trace as the split. Status 4xx or 5xx, a mid-stream error event, a missing or reordered tool_result, a rewritten thinking block, or a tool exception turned into an empty success are integration. end_turn, under the token cap, with the tool results honest, and a total or a field that fails the spec, is model output. Fix that with the instructions, the examples, a schema the response must match, or a retry that includes the validation error. Filing the request_id as an outage does not change the words.

The common mix-up is a parser error on truncated JSON. The parser failed in your process, and the cause is max_tokens. Repair the stop, then parse. The other mix-up is a confident wrong answer with a clean trace. That is not a 500, and a larger timeout will not correct it.

A parser error that is really truncation

An invoice extractor returns HTTP 200. stop_reason is max_tokens. output tokens equal the cap you set, 256. The text ends in the middle of a JSON string. The JSON parser throws, and the alert says the model produced invalid JSON.

The integration received a prefix and treated it as a document. The recovery is to raise max_tokens or continue the assistant message, then parse. The prompt does not need a new instruction about braces until a complete end_turn response still fails the parser.

A tool that failed in silence

The trace shows tool_use for get_invoice, then a user message with that tool_use_id, is_error false, and content "". The database client had thrown a timeout. The catch block replaced the exception with an empty string. Claude answers that the invoice is missing.

The model did what the result said. The integration layer must set is_error true and include the timeout. A second call can then retry the tool or tell the user the lookup failed.

A wrong total with a clean trace

HTTP 200, stop_reason end_turn, 180 output tokens against a cap of 1024, and no tools. The total does not match the line items in the user message. request_id is present because every response has one.

This is model output. Retrying the identical request may sample a different wrong total. The check belongs in your code, and a correction retry should include the failed check. Opening a support ticket with the request id does not change the arithmetic.