LLM Fundamentals

What this skill covers

Domain 5 is Model Selection and Optimization, 16.8% of the CCDV-F exam. This topic is LLM fundamentals, 5.2%: tokens, context windows, sampling, non-determinism, next-token generation, fast mode, extended and adaptive thinking, effort, and zero-shot through multi-shot prompting.

The later topics pick a model, a client, and a bill. This topic is the unit they all share. A token is the thing being generated, cached, and priced. The window is how much of the conversation fits. The sample is one draw from that process.

Next-token generation

A Claude response is built one token at a time. Given the prefix so far, the model assigns a probability to each possible next token and draws one. That token is appended, and the draw repeats until the turn stops: a natural end, your max_tokens cap, a stop sequence, or another stop_reason.

There is no stored answer for the whole prompt that the API looks up. The words you read are the path of those draws. A fact that was likely at the first token can become unlikely after an earlier draw goes a different way. That is why a second call can diverge even when the messages you sent are identical.

Tokens and the context window

A token is the model's piece of text, smaller than a word on average and not a stable fraction of characters. Estimates on the current tokenizer put about 555,000 words in 1 million tokens. Models before the Opus 4.7 tokenizer fit more like 750,000 words in that million. A 200,000-token window is on the order of 150,000 words. Count the string you will send. A word count is a sketch.

The context window is the space for the request and the response being generated. Current Opus 5.5 and Sonnet 5.5 windows are 1 million tokens. Haiku 4.5 is 200,000. max_tokens is a different knob: the most output tokens you will allow on this call. Input plus that output has to fit the window. Filling max_tokens stops the turn with stop_reason max_tokens even when the window still has room.

Thinking tokens are output tokens. They count toward max_tokens and toward the bill, including when the thinking text is omitted and the block comes back with an empty thinking field. A tight max_tokens can end the turn inside the reasoning and leave you no answer text.

Sampling and non-determinism

Sampling parameters such as temperature, top_p, and top_k change which tokens are eligible and how sharply the draw prefers the likely ones. A low temperature concentrates the draw. It does not turn the model into a database. Two calls can still differ. Once they differ at one token, every later token is conditioned on a different prefix.

Whether a given model accepts a non-default temperature is part of that model's API. A rejection is a 400 on the request. The quality of a finished answer is a separate question, answered with an evaluation set rather than with another unchecked sample.

Thinking and effort

Thinking is scratch work before the answer, returned as thinking blocks ahead of the text. The text in the block is a summary, not a raw trace. You pass the blocks back unchanged on later turns, the same way you pass tool_use blocks back. Display summarized shows the summary. Display omitted, the default on many current models, leaves the thinking field empty. The tokens are billed either way.

Extended thinking is the older mode: thinking type enabled plus a budget_tokens cap on the scratch work. It is deprecated on Opus 4.6 and Sonnet 4.6, and later models reject it. Adaptive thinking sets type to adaptive and lets the model decide when and how deeply to think. On Opus 5.5 and several other current models, adaptive thinking is already on. adaptive is a thinking mode. It is not an effort level.

Effort is output_config.effort. The ladder is low, medium, high, xhigh, and max. It steers how much work the response spends, and in adaptive mode that includes how often and how deeply the model thinks. Defaults differ: Opus 5.5 defaults to medium, and several other current models default to high. Haiku 4.5 does not support effort. Not every model accepts every rung. On Opus 5 and later, disabling thinking at xhigh or max returns a 400. Keep thinking on, or drop the effort to high or below.

Use a lower effort on a short classification or a narrow subagent. Use a higher effort when the decision is the product and your evals improve. Changing the top-level effort rewrites the cached prefix, so a conversation that is caching should keep one effort unless the model supports a per-message effort change that leaves the prefix in place.

{
  "model": "claude-opus-5-5",
  "max_tokens": 16000,
  "thinking": { "type": "adaptive" },
  "output_config": { "effort": "medium" },
  "messages": [{ "role": "user", "content": "Check this refund against the policy." }]
}

Fast mode

Fast mode is a research preview that raises output speed, up to about 2.5 times, on supported Opus models on the Claude API. You set speed to fast and send the fast-mode beta. The tokens are the same kind of tokens. The price is a premium schedule, not the model's standard input and output rates. It is not a smaller model and it is not a lower effort.

It is not offered on Amazon Bedrock, Google Cloud, Microsoft Foundry, or Claude Platform on AWS for the Opus models that document the preview. Switching between standard and fast speed starts a new cache prefix.

Zero-shot, single-shot, and multi-shot

Zero-shot is the task and the criteria, with no worked example. Use it when the instruction already names the format and the boundary. A second paragraph of rules is still zero-shot.

Single-shot adds one complete example of input and the output you want. Use it when there is one canonical shape and the model is missing that shape. Multi-shot adds several examples, including the edge you keep seeing in reviews. Examples teach a format and a boundary case more directly than another rule about that case.

Each example sits in the context and on the bill. Put stable examples before the variable user turn so a cache prefix can cover them. An example that contradicts the instruction trains the contradiction. An example is still a sample of the pattern, not a validator. Check the result.

The cap that ate the answer

A proof task uses adaptive thinking and max_tokens 512. The response stops with stop_reason max_tokens. The content is a thinking block and no text. The context window is 1 million tokens and the input was short.

The window had room. The output cap did not, and thinking tokens spend that cap. Raise max_tokens so the answer has room after the scratch work, or lower effort if the evals still pass.

Two routers, one effort

A pipeline classifies a ticket, then decides a refund. Both calls use Opus at max effort. The classifier is a few labels. The decision quotes a policy.

Leave the decision at a high effort if the evals need it. Drop the classifier to low or medium, or move it to a smaller model. The two steps do not need the same rung.