Optimizing AI Context Windows: A Testable Workflow

A coding assistant needs the information that changes its next decision. Sending an entire repository can bury that information; removing too much can make the task impossible. This guide builds a context packet for one task: adding an optional purchase-order reference to an invoice API without breaking existing clients. The example is illustrative, not a report of measured model performance.

Define the protected constraints

Before shortening the input, write the acceptance criteria. For the invoice example, the new field accepts at most 40 characters, remains nullable in storage, and follows the existing response serialization contract. Existing requests without the field must still succeed. A migration must follow the repository's conventions. These are protected constraints: preserve their exact meaning through every revision.

A compact prompt that drops backward compatibility is worse than a longer one that preserves it. Record unresolved behavior as a question to investigate. Do not replace an unknown with a plausible guess merely to shorten the conversation.

Gather a traceable evidence packet

Start with the request validator, invoice model, response serializer, and nearest relevant test. Include file paths and function names so the assistant can request more context precisely. Include the complete relevant function when a fragment would hide control flow or error handling. Add local repository instructions and the installed framework version.

• Task: the desired change and its reason.
• Contract: old and new request examples.
• Evidence: selected source excerpts with their paths.
• Constraints: compatibility, security, naming, and migration rules.
• Verification: observable outcomes that establish success.

Exclude unrelated generated files, dependency lockfiles, and old logs that do not concern the request. Remove credentials before sending source to an external service. If the change crosses an authorization boundary, retain that boundary even when it makes the packet larger.

Compress repetition before meaning

Replace repeated instructions with one clear rule. Summarize the investigation as a decision log: what was tried, what failed, and what remains unresolved. Preserve error messages verbatim when exact wording matters. Keep identifiers unchanged so references remain searchable.

For example: “Add optional purchaseOrderRef to invoice creation. Preserve requests that omit it. Inspect the validator, model, serializer, and creation tests. Verify null, valid string, and overlength input. Follow existing migration conventions.” This replaces a rambling request while retaining its important checks. It does not substitute for the source files.

Avoid mechanically abbreviating every word. Tokenization depends on the model; fewer characters do not always mean fewer tokens. Dense shorthand can introduce ambiguity. Measure size with the provider's tokenizer or actual request usage, and retain readable domain terms unless a shared abbreviation already exists.

Reserve room for tool results and the answer

A context window is a capacity limit, not a guarantee that every included fact will be used correctly. Input, output, and other token budgets vary by model and API. Check the documentation for the model you actually call. Leave room for retrieved files, tool responses, and the planned answer instead of filling the entire window with source material.

If the task still exceeds that capacity, divide it by responsibility: inspect the API contract, plan storage changes, then implement and verify. Carry forward a decision record and exact constraints between stages. Do not assume an earlier conversation will remain available indefinitely.

Compare against a baseline

Run the original and compact packets against the same representative tasks with the same model settings. Record input tokens, output tokens, latency, missed constraints, and whether the proposed changes pass relevant tests. Repeat the comparison when results vary. One successful response does not establish reliability.

For this example, check that an old client can omit the field, a valid reference round-trips, an overlength value is rejected, and the migration follows project conventions. If compression causes a missed requirement, restore the relevant evidence before removing more text. Do not claim cost savings or improved accuracy without measurements.

Recover from missing context

If the assistant invents a helper, provide its real interface. If it changes unrelated behavior, restate the boundary and include the affected test. If it repeats a failed approach, add the failure to the decision log. If essential details keep disappearing, use a larger evidence packet or a smaller task. Compression is an engineering tradeoff, not a guarantee of reasoning quality.

Next, use the context optimizer as an editing aid and compare its result against your protected constraints. Verify tokens and behavior in the actual integration; shorter text alone does not prove preserved meaning.