05 — Context Engineering in Practice: Few-Shot, ReAct, Compaction, and the Rest

Few-shot, ReAct, chain-of-thought, compaction, function calling, caching, streaming — these used to sit in my head as unrelated prompting tricks. The pattern that organizes all of them: every technique here is a different way of deciding what goes in the context window, in what shape, and for how long. [1] None of them are new kinds of model. They're all ways of engineering the payload the model sees.

The framing that finally landed is a single question I now ask about every technique: _what is this doing to the context window?_ Few-shot adds examples. Chain-of-thought makes the model write its own scratchpad into the context. Compaction shrinks what's there. Function calling injects a tool schema. Caching avoids recomputing the same prefix. Streaming changes how the output comes back, not the input at all. Once each technique maps to a clear effect on the window, choosing between them stops being a matter of vibes.

Zero-shot versus few-shot: whether to seed examples

The most basic decision is whether to give the model examples at all. Zero-shot relies purely on the model's pretraining — describe the task, let it infer what to do [2]. Few-shot puts a small number of input–output pairs into the prompt so the model can imitate the pattern [3].

The tradeoff is direct and lives in the context window:

• Zero-shot is lean. It costs no extra tokens and adapts to any task. It works when the task is unambiguous and the model has seen similar things in pretraining.
• Few-shot is heavier but more reliable for idiosyncratic formats. It teaches the model the exact output shape by demonstration. The cost is tokens — every example eats into the budget — and brittleness if the examples don't cover the variation in real inputs.

My rule: start zero-shot. Reach for few-shot only when the output format is unusual or the model keeps drifting, and use the fewest examples that stabilize the behavior.

RAG and dynamic filters: retrieval as context

Retrieval-Augmented Generation (RAG) is the biggest context-engineering lever there is. Instead of stuffing knowledge into the prompt by hand or fine-tuning it into the weights, I retrieve the relevant chunks from a knowledge base at request time and place only those in the window [4]. The context is dynamic — it changes per query — which is the whole point: the model sees fresh, task-specific evidence instead of static training data.

Dynamic filters sharpen this further. They pre-filter what gets retrieved based on the query, the user, or metadata — so the model receives only the pertinent context, not everything vaguely related [5]. Without filters, RAG tends to drown the model in "mostly relevant" passages that compete for attention. With them, the window stays lean and the signal stays high.

Chain-of-thought: the model's own scratchpad

Chain-of-thought (CoT) is a quietly powerful technique. Instead of asking for the final answer directly, the prompt asks the model to write out its intermediate reasoning first [6]. Those intermediate steps end up _in the context window_, and the model then conditions on its own written reasoning when producing the final answer.

Why this works: the model is, at heart, a next-token predictor. Asking it to produce the final answer in one shot forces all the reasoning to happen implicitly. Letting it write steps externalizes that reasoning into the context, where each subsequent token can build on what was already written. For multi-step problems — math, logic, code analysis — CoT reliably improves accuracy and makes the answer auditable [6]. The cost is tokens and latency, which is why I reserve it for problems that genuinely need decomposition.

ReAct: reasoning plus action in a loop

ReAct (Reason and Act) extends chain-of-thought into a loop that calls external tools [7]. The pattern is a strict cycle:

1. Thought — the model reasons about the current state.
2. Action — it emits a call to an external tool or API.
3. Observation — the tool's result is appended to the context.
4. Repeat until the model can produce a final answer.

ReAct is what turns an LLM from a text generator into an _agent_. The context window fills with the trace of thoughts, actions, and observations — which is exactly the engineering challenge. Long ReAct runs bloat the window fast, which is why compaction (next) exists.

Context compaction: keeping the window lean

When a session runs long — a long conversation, a multi-step agent run — the context window fills with content that's no longer all relevant. Context compaction shrinks what's there without losing the signal: summarizing earlier turns, dropping spent observations, re-ranking what remains [8].

Compaction is the technique I underrated for the longest time. It's not a nicety; it's a hard requirement for long-running agents. Without it, the window overflows (the model truncates and forgets the actual task) or the cost explodes (every token is paid for on every turn). The art is deciding what to summarize versus what to keep verbatim — usually: summarize history, keep the latest user turn and any tool output that's still load-bearing.

Function calling: tools as context

Function calling lets the model decide to invoke a predefined function, emitting structured arguments instead of free text [9]. From a context-engineering view, what's happening is two things at once:

• A tool schema is injected into the context up front, describing the available functions.
• When the model decides to call one, its structured output is added to the context, then the function's return value is appended.

Function calling is the mechanism that makes ReAct-style agents practical at scale — the model doesn't have to produce free-text tool calls and hope they parse. The schema-constrained output is reliable, and that reliability is what lets me wire real APIs behind it.

Prompt caching: not recomputing the prefix

Many production prompts share a long stable prefix — the system prompt, the tool schemas, a fixed knowledge preamble. Prompt caching stores the processed form of that prefix so it isn't recomputed on every call [10]. The effect is dramatic on cost and latency for applications with a stable instruction core, because the cached prefix is processed once and reused across requests.

Caching is purely a performance optimization — it doesn't change what the model sees, only how cheaply it sees it. But for any feature that takes real traffic, it's often the difference between a sustainable bill and an unsustainable one.

Streaming responses: how the output returns

Streaming isn't about the input context at all — it's about how the output arrives. Instead of waiting for the whole response, the model starts sending tokens as soon as it generates them, and the UI grows in real time [11].

The engineering tradeoff is real. Streaming feels fast to users and lets them interrupt early, which is why chat-like interfaces all use it. Non-streaming is simpler — easier to cache, easier to log, easier to post-process — and is fine when the output is short or when downstream steps need the whole thing before starting. I default to streaming for any user-facing chat, and non-streaming for batch or programmatic calls.

How I use this

I treat these techniques as a stack I assemble per feature, not a menu I sample from. For a simple Q&A feature, zero-shot plus a good system prompt is enough. For a knowledge-grounded assistant, I add RAG with dynamic filters. For anything that needs external data or multi-step reasoning, I reach for function calling and ReAct, with compaction waiting in the wings the moment runs get long. Caching goes in once traffic justifies it. Streaming goes in if a human is reading the output. The discipline is adding each layer only when the previous one proves insufficient — because every technique costs tokens, latency, or complexity, and the cheapest context is the one I didn't have to build.

References

[1] LangChain, "Context Engineering for Agents," 2025. [Online]. Available: https://blog.langchain.com/context-engineering-for-agents/

[2] IBM, "What is zero-shot prompting?," 2024. [Online]. Available: https://www.ibm.com/think/topics/zero-shot-prompting

[3] IBM, "What is few shot prompting?," 2024. [Online]. Available: https://www.ibm.com/think/topics/few-shot-prompting

[4] O. Lewis, "4 context engineering strategies every AI engineer needs to know," 2025. [Online]. Available: https://newsletter.owainlewis.com/p/4-context-engineering-strategies

[5] Towards Data Science, "Beyond RAG: The Rise of Context Engineering and Semantic Layers for Agentic AI," 2025. [Online]. Available: https://towardsdatascience.com/beyond-rag/

[6] J. Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," arXiv:2201.11903, 2022. [Online]. Available: https://arxiv.org/abs/2201.11903

[7] Prompting Guide, "ReAct Prompting," 2024. [Online]. Available: https://www.promptingguide.ai/techniques/react

[8] "Context Compaction," gist, 2025. [Online]. Available: https://gist.github.com/badlogic/cd2ef65b0697c4dbe2d13fbecb0a0a5f

[9] The New Stack, "A Comprehensive Guide to Function Calling in LLMs," 2024. [Online]. Available: https://thenewstack.io/a-comprehensive-guide-to-function-calling-in-llms/

[10] IBM, "What is Prompt Caching?," 2024. [Online]. Available: https://www.ibm.com/think/topics/prompt-caching

[11] A. Gil, "AI for Web Devs: Faster Responses with HTTP Streaming," 2024. [Online]. Available: https://austingil.com/ai-for-web-devs-streaming/