← Home · Templates · Linter · Debugger

How to write a system prompt that actually works

Most prompt failures are structural — missing role, missing format spec, no fallback. Fix the structure first. The content follows.

The four components

Every working system prompt has the same four parts. Missing any one of them forces the model to fill the gap with its defaults, which are inconsistent across calls and across models.

01
Role definition
One sentence: what the model is and what it is not. Be specific about the domain, the user, and the task. Generic roles ("You are a helpful assistant") produce generic behavior.
Without this: the model role-plays its training defaults, which shift unpredictably.
02
Output format
Specify exactly how the response should be structured. If you need JSON, write the schema. If you need markdown headings, name them. If you need prose, say so and give a length target.
Without this: the model chooses its own structure, which drifts between calls and breaks downstream parsing.
03
Constraints
Hard limits on what the model must not do. Keep this list short — three or four items at most. More than that usually means the role definition was insufficient.
Without this: edge-case inputs produce unpredictable behavior. The model improvises.
04
Fallback instruction
What the model does when the user's request is out of scope, ambiguous, or unsafe. "If you cannot answer, say X" is enough. Without it, the model either refuses in a variety of ways or hallucinates an answer.
Without this: out-of-scope inputs produce inconsistent refusals or confident hallucinations.

The workflow — three steps

There's a natural sequence for building a system prompt that works in production. Each step catches a different class of problem.

1. Start from a template
→
2. Lint before you ship
→
3. Debug after it breaks
Step 1
System Prompt Templates →
12 production-ready templates for the most common agent types: customer support, code reviewer, JSON extractor, SQL generator, and more. Each has the four structural components already filled in.
Use when: starting from scratch or looking for a reference structure.
→
Step 2
System Prompt Linter →
Paste your prompt and check it for structural issues before deployment. Catches missing role definition, missing output format, unfilled placeholders, weak instruction language, and more.
Use when: you've written a prompt and want to check it before shipping.
→
Step 3
AI Prompt Debugger →
Pick your model (Claude, GPT-4, Gemini), select your symptom, answer one question, get a specific fix with copy-paste snippets. Covers 21 failure patterns with model-specific diagnosis.
Use when: your prompt is deployed but producing wrong results.
→

Common structural mistakes

These account for the majority of system prompt failures in production. The linter catches most of them automatically.

Vague role: "You are a helpful AI assistant." The model has no scope, no user, no domain. Every response is a guess about what "helpful" means in this context.
No output format: The model produces markdown in one call, plain text in another, and a numbered list in a third. Downstream parsing breaks intermittently and the bug is hard to reproduce.
Conflicting instructions: "Be concise" followed by "explain your reasoning in detail." The model resolves conflicts in unpredictable ways — sometimes prioritizing the first instruction, sometimes the last, sometimes ignoring both.
Overly permissive constraints: "Try not to make things up." A soft instruction doesn't override a strong user request. Hard constraints require language like "never" or "you must not" — not "try" or "aim to."
No fallback: The prompt handles the happy path but not ambiguous or out-of-scope inputs. When a user asks something unexpected, the model either refuses unhelpfully or invents an answer.
Unfilled placeholders: [COMPANY_NAME] or [YOUR POLICY HERE] left in the live prompt. The model treats these literally or ignores them — both cause incorrect behavior.

Frequently asked questions

What is a system prompt?
A system prompt is the instruction set you pass to an LLM before the user's first message. It defines the model's role, behavior, output format, constraints, and fallback handling. Unlike the user message, the system prompt is usually invisible to the user and persists across the conversation.
Why do system prompts fail?
Most system prompt failures are structural, not semantic. The four most common gaps: no explicit role definition, no output format specification, no constraints on out-of-scope requests, and no fallback instruction for ambiguous inputs. The model fills in each gap with its defaults, which are inconsistent.
What should a system prompt always include?
Four components: (1) a specific role statement — what the model is and is not; (2) an output format specification — JSON schema, heading structure, or prose style; (3) constraints — what the model must not do; (4) a fallback — what the model does when the user request is out of scope or ambiguous.
How long should a system prompt be?
Long enough to specify all four structural components; short enough that a developer can read it in under two minutes. Most production prompts are 150–600 words. Longer prompts tend to accumulate contradictions and dead weight that reduce consistency.
Should I use negative instructions (don't do X) in a system prompt?
Yes, sparingly. Negative instructions are useful for hard constraints — things the model must never do regardless of context. Too many negative instructions signal that the positive role definition was insufficient. If you have more than three or four, rewrite the role definition instead of adding more negatives.
How do I debug a system prompt that isn't working?
Start with the structural checks: is the role specific? Is the output format explicit? Is there a fallback? If the structure is sound, the problem is usually one of three things: the constraint is too vague, the output format description conflicts with an example, or the fallback instruction is missing. The AI Prompt Debugger walks through the most common symptoms with model-specific fixes.
Does the system prompt get overridden by the user message?
System prompt instructions have higher precedence than user message instructions in most models. However, very insistent user messages can shift model behavior — which is why constraints in the system prompt need to use strong language ("you must not," "never") rather than soft language ("try not to," "avoid").