6.1 Context Engineering
- The window holds instructions, tools, history, tool results, thinking, and the new output.
- Bloat is unused text you keep resending. Drift is the model following that text instead of the task.
- Prune a tool result to the fact once the next step no longer needs the dump.
- Server-side clearing drops the oldest tool results past a threshold and leaves a placeholder.
- Clearing changes the cached prefix. Clear a meaningful number of tokens.
- A compaction block replaces everything before it. Send the block back on the next call.
- Default compaction trigger: 150,000 input tokens. Minimum trigger: 50,000.
- Custom compaction instructions replace the default summary prompt.
- Keep a verbatim tail by leaving those turns out of the summary and placing them after the block.
- A subagent isolates the search. The parent gets the result you asked for, not the child's log.
| If the question says... | The answer is likely... |
|---|---|
| "the policy is in the system prompt and the model quotes the log" | Prune the log. The context drifted |
| "tool results from ten turns ago" | Clear or replace them. Keep the finding |
| "input tokens crossed the compaction trigger" | Summarize the old span and send the block back |
| "the last user correction must stay exact" | Keep that tail after the compaction block |
| "a file review is filling the parent" | A subagent per file, result only |
| "compaction ran and the next call forgot the summary" | The block was not included in the next messages |
| "we added a sentence to the summary instructions" | Custom instructions replace the default prompt |
| "clearing one token to save the cache" | The clear itself misses the cache. Clear enough to matter |
| Trap | Correct answer |
|---|---|
| A bigger model fixes drift. | Drift is the bytes in the window. Remove the competing text. |
| Compaction keeps every earlier message and adds a summary. | Content before the block is dropped once the block is sent back. |
| A subagent already knows the parent chat. | Its window starts from the task you send. |
| Tool pruning means deleting the tool. | The tool stays. The old result body is what goes. |
| Any clear is a free saving. | A clear invalidates the cached prefix. Size the clear so the miss pays. |
6.2 Prompt Engineering
- Name the task, the decision rule, and what to do when the source is silent.
- System prompt: rules that survive the next call. User turn: this case.
- Do not put customer data or retrieved pages in the system prompt.
- Delimit untrusted text and tell the model it is data.
- Few-shot examples show an edge the instruction already describes. Put them before the live case.
- An example of an invented field teaches invention.
- Constrain shape with fields, enums, length, or a schema.
- Change one prompt element at a time and score the same set.
- A full window of logs is a context problem. More adjectives will not fix it.
- Bound the length of user fields so one paste cannot occupy the window.
| If the question says... | The answer is likely... |
|---|---|
| "the policy must hold on every ticket" | System prompt |
| "this customer's message" | Delimited user content |
| "the format fails and the rule is already specific" | Add few-shot examples |
| "the model fills a date the receipt lacks" | Tell it to mark the date unknown |
| "three edits and a new model" | Score one change |
| "ignore previous instructions inside the page" | The page stays data. The rule stays in system |
| "be concise" | A counted constraint, or a schema |
| "the id is in the system prompt for convenience" | Move the id to the user turn |
| Trap | Correct answer |
|---|---|
| A longer system prompt is always clearer. | Variable text in the system prompt is data in the instruction slot. |
| Few-shot means repeating the instruction three times. | Few-shot means complete input and output examples. |
| Putting the document in the system prompt makes the model follow the policy. | The policy is the instruction. The document is the user case. |
| Sanitization means the model cannot obey text it can see. | It keeps that text out of the instruction slot. You still check the action. |
| Refine by shipping every idea that might help. | One change, the same set, then keep or drop it. |
6.3 Output Handling
- Structured output is a schema on the response or strict checking on tool inputs.
- Use output_config.format for JSON. A prompt sentence is a weaker constraint.
- Nullable fields are how you allow absence. Required fields pressure the model to fill them.
- Parse, then apply business rules: sums, caps, known ids.
- A schema-valid object can still be wrong.
- On failure, retry with the specific error only when the source contains the fact.
- Do not parse a thinking block or a max_tokens prefix as the answer.
- Do not eval or exec model text.
- A confident citation counts when you find it in the source, not when it sounds specific.
- The model is not the system of record. The action sits behind your check.
| If the question says... | The answer is likely... |
|---|---|
| "JSON.parse succeeded" | Still run the business check |
| "the date was required and the receipt has none" | Make the field nullable. Do not retry for a date |
| "lines sum to 450 and the total is 500" | Reject and retry with that discrepancy |
| "the JSON ends mid-string and stop_reason is max_tokens" | Finish the turn, then parse |
| "the model says it is very confident" | Ignore the tone. Check the source |
| "a citation the prompt never contained" | Drop it or verify it outside the prose |
| "tool input is not strict" | Validate before you run the tool |
| "the same prompt, second try" | A new sample, unless you included the error |
| Trap | Correct answer |
|---|---|
| Structured output means the facts are correct. | It means the shape matches the schema. |
| A required field is safer than a nullable one. | Required fields invent values when the source is silent. |
| Asking the model if it is sure is a validation. | Validation is your rule or a comparison to the source. |
| Repair truncated JSON by adding braces. | A max_tokens prefix is unfinished. Continue the turn. |
| Fluent text with a section number is a citation. | The section has to appear in the source you trust. |