I’ve spent the last two years shipping AI features across two production codebases — one .NET, one TypeScript — and teaching engineers how to build them. Somewhere around month eighteen, the thing I was actually doing every day stopped looking like prompt engineering and started looking like something else.
Other people noticed too. “Context engineering” has been quietly replacing “prompt engineering” in serious AI-engineering discourse for the last twelve months, and in 2026 it’s the meta-topic of the field. This is the post I wish someone had handed me on day one: not a tutorial, a frame.
The thesis is simple. Prompt engineering was always a misnomer. The work was never about writing clever words. It was about deciding what the model gets to see when it runs — and what it doesn’t. That’s a systems-design problem. And it’s the only skill that scales when you go from a demo to a product.
Why “prompt engineering” was the wrong name
A prompt is one variable in a very long expression.
When a real production feature calls a model, the input includes the user’s message, yes — but also a system prompt, the schema of every tool the model can call, a slice of conversation history, a freshly retrieved set of documents, summaries of older turns the model is supposed to remember, the user’s profile, the current task state, possibly the output of a planner step that ran two seconds ago, and a set of constraints the legal team wrote down in a markdown file three months ago. The user’s actual sentence is maybe five percent of what the model is reading.
The other ninety-five percent is what determines whether the feature works.
Calling that “prompt engineering” is like calling a database query language “string formatting.” It misses the entire problem. The interesting questions aren’t how do I phrase this? They’re what should be in this window? what shouldn’t? where does it come from? when is it stale? how much of it can I afford?
Those are engineering questions. They have a name now: context engineering.
What context actually is
Useful frame: context isn’t one thing. It’s at least five distinct layers, and each layer has its own failure mode when you get it wrong.
| Layer | What it is | Failure mode if you get it wrong |
|---|---|---|
| System & role | The persistent identity, role, and constraints the model operates under | Model drifts, ignores rules under pressure, follows the user past your safety rails |
| Tools | Functions the model can call, with their schemas and descriptions | Wrong tool fires, infinite loops, schema injection, security holes |
| Retrieved knowledge | Documents, chunks, and facts injected per query (RAG) | Stale answers, confident hallucinations, latency, cost spikes |
| Conversation / agent history | What’s already been said, what’s already been done, what’s been decided | Lost state, repeated work, contradictions, gaslighting |
| Task & user state | The current goal, the user’s data, the environment the model is acting in | Generic answers, no personalization, missed obvious context |
When something goes wrong in a production AI feature, it almost never goes wrong because the prompt was phrased badly. It goes wrong because one of these five layers leaked, was stale, was missing, or was too big. Engineers who can’t tell which layer broke are the engineers still describing themselves as prompt engineers.
The four hard problems of context engineering
The discipline boils down to four problems that show up on every non-trivial AI feature. None of them is solvable with better wording. All four are systems-engineering problems.
Selection: what goes in?
The context window is finite. Even with two-hundred-thousand-token models, you can’t shovel your entire corpus into every call — not because you’d hit the limit, but because the model’s attention dilutes across irrelevant material and your bill goes vertical.
Selection is a retrieval-engineering problem. Hybrid BM25 plus dense embeddings, reranking, query rewriting, freshness boosts, deduplication, source weighting. Picking the right eight thousand tokens out of a two-hundred-thousand-token corpus is harder than picking the right index for a SQL query, and you get it wrong in more interesting ways.
The teams that ship reliable AI features treat retrieval as a first-class subsystem with its own metrics, evals, and on-call. The teams that don’t ship them ad-hoc and wonder why answers degrade silently.
Compression: how to fit more without losing fidelity
Long agent runs accumulate context faster than the window can hold it. The naive answer is to truncate. The grown-up answer is to summarize — and summarize again, recursively, at structured checkpoints.
This sounds simple. It isn’t. Every summarization step is lossy, and the loss compounds. The decision of what’s worth keeping in the summary is exactly the kind of decision the model is bad at making unsupervised. Get it wrong and the agent forgets, mid-task, the constraint it was given five minutes ago.
The shape that works in practice: structured state — facts, decisions, open questions, current goal — written and updated by the agent itself but on a schema you control. Free-form summaries decay. Structured state survives.
Routing: which context belongs to which step
Multi-step agents don’t need the same context in every step. A planning step needs the full goal and the available capabilities; a tool-execution step needs only the slice relevant to this tool call; a synthesis step needs the planning trace and the tool outputs.
If you give every step the union of everything, you pay for it in tokens, latency, and quality. Models lose acuity in long, undifferentiated windows. They sharpen in focused ones.
Routing context per step looks far more like system design than it does like prompt design. You’re deciding which subsystem gets which view of state. That’s an architecture problem.
Eviction: what to drop, and when
Forgetting is a feature. The hard part isn’t remembering — it’s deciding what’s safe to forget.
In practice you end up with tiered memory: hot context that’s in the window right now, warm context that’s summarized and retrievable on demand, cold context that’s archived and only pulled if explicitly referenced. Tool results follow the same pattern — full output in the immediate next turn, truncated to a digest the turn after, dropped to “tool X was called at T with result Y-shaped” three turns later.
This is the part most teams skip until their agent runs start costing thirty cents apiece and taking ninety seconds.
The mental model: context as a build artifact
Here’s the shift that changes how senior engineers think about this.
Stop thinking of the prompt as a string you write. Start thinking of the context as a build artifact your runtime produces, every turn, from many sources:
[user input]
[conversation summary] ──┐
[retrieved docs] │
[tool schemas] ├──► context assembler ──► model
[system rules] │
[task state] │
[recent tool outputs] ──┘The context assembler is a piece of software you own. It has inputs, transformations, caching, observability, and tests. It’s the most important component in your AI feature, and on most teams nobody is officially responsible for it.
The prompt is the last one percent of a pipeline that’s ninety-nine percent systems engineering.
Once you see it this way, the rest of the discipline falls out naturally. You version the assembler. You eval it. You profile its token output. You write integration tests that pin specific context shapes. You instrument every layer so when quality regresses you can see which layer changed. None of this is exotic — it’s normal software engineering, applied to the layer the prompt engineers never noticed they were standing on.
Five rules I’ve learned in production
Numbered because they’re earned, not derived.
1. Treat the context window like a memory hierarchy. Hot is what’s in the window now. Warm is summarized and retrievable. Cold is archived. Move things between tiers deliberately — based on relevance, age, and cost — not by accident. Most context bugs are tier-management bugs in disguise.
2. Measure tokens like you measure latency. Tokens are now a load-bearing performance metric. Every feature should have a token budget per turn, per session, per user-day. Without observability on this, your costs and your latency creep up monotonically and you find out from finance, not engineering.
3. Never trust the model to manage its own state. Models will confidently tell you they remember things they don’t, and will silently drop constraints they were given fifteen turns ago. State management is your job. The model executes; it doesn’t bookkeep.
4. Separate planning context from execution context. A planner sees the goal and the toolbox. An executor sees the current sub-goal and the relevant slice. A synthesizer sees the trace and the outputs. Mixing these three views in one window is the most common source of quality regressions I’ve seen in agent systems.
5. Eval the pipeline, not the prompt. If your evaluation harness only A/B-tests prompt wording, you’re optimizing the last one percent. Build evals that vary retrieval recall, summarization fidelity, tool-output shape, and history depth. That’s where the actual quality variance lives.
Where prompt engineering still matters
Honest nuance, because the contrarian frame is a frame, not the whole truth.
Prompt-level wording still dominates in:
- Single-turn classification and extraction. Short, no tools, no history, no retrieval. Wording is the variable. Treat this case as the special case it is.
- Structured outputs with tight schemas. Function-call routing, JSON-mode coercion, schema enforcement. Wording moves the needle here in ways context cannot.
- Tone, persona, and refusal behavior. Tone is largely a prompt concern. Refusals are a prompt-plus-system concern. Persona is almost entirely prompt-shaped.
But anything agentic, anything retrieval-backed, anything multi-step, anything that carries state across turns — context engineering wins. And in 2026, that covers most of what production AI actually looks like.
What this means for your career in 2026
If you’re a mid-level engineer building AI features and you want to know what to invest in next, here’s the short version.
The job description for a senior AI engineer in 2026 isn’t “prompt engineer.” It’s “context engineer.” The skills that compound are: retrieval design, evaluation harnesses, observability for token economics, agent orchestration, schema design for tool calls, summarization strategy, and the discipline to treat the context window as a controlled environment rather than a wishlist.
The skills that are quietly depreciating are: clever prompt incantations, prompt-template libraries treated as products, “prompt patterns” frameworks, and the assumption that better wording will rescue an under-engineered pipeline. None of these are useless. All of them are smaller than they were two years ago.
The 80/20 if you read nothing else: pick one production AI feature you own. Map its context pipeline end-to-end on a whiteboard — every source, every transformation, every tier. Instrument the token cost of each layer. Then build a five-case eval suite that varies one layer at a time. You will discover, in about a week, that what you thought was a prompt problem was a retrieval problem, or a state-management problem, or a routing problem. That discovery is the entire skill.
The deeper point
The shift from prompt engineering to context engineering is the same shift that happened when “web programming” became “systems design,” and when “writing SQL” became “data engineering.” The interesting work moved from the surface — the words, the queries, the strings — to the pipeline underneath.
The design-time companion to this discipline is spec-driven development — how teams actually produce the inputs that drive these context pipelines in the first place. That’s the next post.
If you want to go deeper, the courses below cover the parts of this discipline I teach most often: Prompt Engineering & AI Workflow Automation for the prompt-layer foundations, Building LLM-Powered Apps: RAG & Agents for retrieval and multi-step orchestration, Building with Claude API: Production AI Apps for the runtime side, and Building Agents with the Claude Agent SDK for the full agent surface where every problem in this article shows up at once.
