12 — Prompt Engineering — Writing the Spec for a Probabilistic System

"Just talking to the model" was my prompt-engineering model, which made every bad output feel like a machine failure. The reframe that stuck: a prompt is a spec for a probabilistic system. An LLM doesn't look up answers — it guesses what comes next — so the quality of what comes back is capped by how clearly I described what I wanted [1]. "Vague in, vague out" isn't the model failing; it's the model guessing the most common version of an unclear request.

The image that clicked for me: treat a prompt like a brief you hand to a smart-but-literal junior. They're capable and know a lot, but they can't read your mind. If I don't spell out the tools, the tone, and the shape of the answer, they'll fill the gap with whatever answer is most common in everything they've read — not what I actually wanted. Good prompting is just writing that brief on purpose.

The model under the hood: predicting the next token

Before the techniques, the one fact that makes them all make sense. An LLM is, at its core, a next-token predictor — given the text so far, it assigns a chance to every possible next piece and picks one [2]. A token is just a small chunk of text, roughly a piece of a word. There's no step where the model goes and looks up the real answer; the reply is built piece by piece, each one chosen from a list of possibilities. That's why the same prompt can come back slightly different each run.

The thing I had to internalize: every word in my prompt is a clue the model uses to guess. My job is to pile the odds onto the kind of answer I want, instead of letting them spread across every possible reading. Every technique below — being specific, giving examples, fixing the format — is just a way to pile the odds in my favor.

Zero-shot vs. few-shot: showing, not telling

The first fork in prompting is whether you give examples [3].

• Zero-shot — ask with no examples. "Translate to Indonesian: good morning." Works for common, well-trodden tasks where the model already has a strong habit to lean on. Cheap and fast.
• Few-shot — include a handful of input→output pairs before the real ask. The model infers the pattern from the examples and continues it.

Few-shot is what demystified the whole thing for me. Instead of describing a format in prose and hoping the model reads my description right, I just show the format a few times. The model is doing what it's built for — continuing a pattern — and a concrete example is far less ambiguous than a paragraph of instructions. It's the same instinct as reaching for a working test case instead of a spec document: the example is the spec.

The trade-off is length. Examples eat up the model's memory and cost money, and if they're too narrow they can pin the model too hard. I reach for zero-shot first; if the answer drifts, I add two or three examples before I touch the instructions.

Being specific is the whole game

The failure I hit most often is leaving things unsaid. "Write a function to validate an email" sounds clear to me, but it leaves a dozen decisions to the model's habits: which language, which rules count as "valid," what to hand back when it's wrong, what about non-English email addresses, do we return a value or throw an error? Each of those is a guess the model makes by leaning on the average of its training.

The fix isn't more words — it's pinning down the decisions that matter:

• Role — "You are a senior TypeScript engineer." This sets how formal the answer sounds and how much it assumes I already know.
• Context — "This runs in a Next.js Server Component, so the browser isn't available." The model can't respect a limit it was never told about.
• Constraints — "Pure function, no outside libraries, throw on bad input, return a yes/no value." Each one closes off a whole direction the answer could wander.
• Output format — "Reply with only the code, no small talk, no markdown wrapping." Otherwise the model happily pads everything with a chatty explanation, because that's the most common shape of its training data.

I noticed these line up almost one-to-one with the sections of a good bug ticket or code review: who, where, what, in what shape. The skill carries over — prompt engineering rewards the same clear-thinking muscles as writing specs for people.

Constraining the output

Telling the model what shape the answer should take is where I get the biggest payoff for the least effort. Without a format constraint, the model falls back to whatever is most common in its training — usually a friendly, padded explanation. For a coding task, that's noise.

Constraints I use all the time:

• "Return only valid JSON matching this exact shape." Then I write the shape down.
• "No preamble, no explanation. Code only."
• "Use exactly these headings: Problem, Approach, Trade-offs."
• "Maximum 3 sentences."

When the output is going to be read by code — like an LLM call inside a function — I'm strict about it. A stray "Here's your JSON:" at the start will break the parser that reads it. Format constraints aren't politeness; they're part of the contract between me and the model.

The cost of ambiguity

The single idea that reframed everything: ambiguity has a measurable cost. Every decision I leave unsaid becomes a guess drawn from the model's habits, and each one is a chance for the answer to drift from what I meant. A prompt with five un-pinned decisions is rolling five dice; the result lands "plausible" on every one, but rarely "what I meant" on all of them at once [1][3].

This is why the same prompt gives inconsistent results across runs, and why two people writing "the same" prompt get different answers — they almost certainly didn't write the same prompt, because the things they assumed but didn't say differ. Tightening a prompt isn't about being polite to the model; it's about closing the gaps where my meaning could be read two ways. When a prompt "isn't working," my first move now is to hunt for the decision I left silent.

How I use this

The habit these notes left me with is a single pause before I send: I read my own prompt back and ask, what did I leave to the model's imagination? If there's a role, context, constraint, example, or output format I assumed but didn't write, I add it. Most of the time the prompt that "didn't work" was just a brief with holes in it — and the model faithfully filled them with the average of everything I didn't say. Writing the spec on purpose, the way I'd write one for a person, is the whole skill.

References

[1] learnprompting.org, "Introduction to Prompt Engineering," 2024. [Online]. Available: https://learnprompting.org/courses/intro-to-prompt-engineering

[2] IBM, "Prompt engineering techniques," IBM Think, 2024. [Online]. Available: https://www.ibm.com/think/topics/prompt-engineering-techniques

[3] learnprompting.org, "Prompt Engineering," 2024. [Online]. Available: https://learnprompting.org/

[4] roadmap.sh, "Prompt Engineering Roadmap," 2024. [Online]. Available: https://roadmap.sh/prompt-engineering