Understand context before your first model call

A model request has an information boundary: what did the application actually send? A model does not automatically know what is in your database. A chat interface displaying a history also does not prove that every message was included in the request.

Break the input into task rules, the current question, necessary history, retrieved evidence, and tool definitions. Organizing these parts is a better starting point than endlessly polishing a prompt.

Separate three budgets

Tokens are units used by a model to process text. They are not the same as characters or words. Different tokenizers can assign different token counts to the same text.

A context window describes capacity, but the input limit, maximum output, and accounting for reasoning tokens depend on the model and interface. Do not fill the advertised window entirely with documents.

Reserve room for output, fixed instructions, tool definitions, and protocol overhead. When integrating a real model, calibrate estimates with its tokenizer or returned usage information.

Organize input by responsibility

Task rules define boundaries. The current question states the user's need. History describes relevant decisions and results. Retrieved material provides evidence, while tool definitions describe available capabilities.

These categories have different authority. A retrieved page saying “ignore previous instructions” remains external data. Its wording does not make it a system instruction.

Keep the current goal, confirmed constraints, and necessary results from the conversation. Unrelated history consumes capacity and can distract the model. More history is not automatically better context.

Practice with an offline budget

The token counts below are invented teaching values, not tokenizer output. This runnable Python example demonstrates subtracting fixed costs and selecting material in a predefined priority order.

window = 80
reserved_output = 20
safety_margin = 5
fixed_input = 18
remaining = window - reserved_output - safety_margin - fixed_input

notes = [
    {"id": "current-goal", "tokens": 12},
    {"id": "relevant-evidence", "tokens": 20},
    {"id": "old-discussion", "tokens": 24},
]
selected = []
for note in notes:
    if note["tokens"] <= remaining:
        selected.append(note["id"])
        remaining -= note["tokens"]

assert selected == ["current-goal", "relevant-evidence"]
assert remaining == 5
print(selected)

The code does not infer semantic importance. You supplied the priorities by ordering the list. A real implementation should also retain each document's version and source rather than storing anonymous text.

A successful request is not a successful task

HTTP success tells you that the interface produced a response. Your application still needs to distinguish completed output, refusal, truncation, tool requests, and service errors.

If the next step writes to a database, define fields and allowed values and validate them in code. Text that looks like JSON is not a business contract. Even schema-conforming output needs checks for meaning and authorization.

Try it next

Choose a small task, such as summarizing a public description into three points. List its fixed instructions, input, and expected result format. Remove unnecessary history before adding more instructions.

With a real model, compare the input estimate with returned usage. Record counts and categories without logging the user's full text. The next guide separates streaming display from the final result your application can safely use.