The trick that stops working
Everyone starts with prompt tricks: "think step by step", "you are an expert", "be concise". They help a little, then they stop mattering. The moment you put a model into a real product, the question is no longer how you phrase one sentence — it is what information sits in the window when the model answers.
That work has a better name: context engineering. It is the discipline of deciding what the model sees, in what order, and at what cost.
Three things competing for the window
Every request fights over a fixed budget of tokens:
- Instructions — the rules, the format, the role.
- Knowledge — retrieved documents, code, prior messages.
- Room to answer — the output you actually want back.
Stuff the window with knowledge and you starve the answer. Pad it with instructions and you crowd out the facts. Most bad LLM output is not a reasoning failure — it is a packing failure.
Relevance over volume
The instinct is to give the model everything "just in case". This backfires. Irrelevant context is not neutral; it actively distracts. A focused 2,000-token prompt routinely beats a bloated 30,000-token one, because the model spends its attention on signal instead of noise.
// Worse: dump the whole file
const ctx = await readEntireRepo();
// Better: retrieve only what this question needs
const ctx = await retrieve(question, { topK: 6, maxTokens: 2000 });
Order matters more than you think
Models weight the start and end of the window more heavily than the middle. Put the instruction that must not be ignored last. Put the grounding facts near the question they answer. Treat the prompt as a layout problem, not a paragraph.
The takeaway
Stop hunting for the magic phrase. Build a pipeline that assembles the right context for each request, trims what does not help, and leaves the model room to respond. The prompt is the last 5%. The context is the work.