Stop Prompting, Start Programming with DSPy
After reading about DSPy in Pedram Navid’s excellent blog post, I wanted a smaller example that showed what the framework changes in practice.
The example takes a movie review, classifies it as Positive or Negative, and gives a short reason.
Without DSPy: prompt and parser stay coupled
The manual version couples application logic to a prompt string. It also handles the API call, formatting instructions, cleanup, and JSON parsing itself.
# --- THE OLD WAY ---
import openai
import json
# 1. The Brittle Prompt String
# If you change a word here, you might break the JSON parser later.
PROMPT_TEMPLATE = """
Analyze the sentiment of the following movie review.
Return ONLY a JSON object with keys: "sentiment" and "reason".
The sentiment must be exactly "Positive" or "Negative".
Do not include markdown formatting like ```json at the start or end.
Review: "{review_text}"
"""
def analyze_review_manual(review_text):
# 2. Manually construct the prompt
prompt = PROMPT_TEMPLATE.format(review_text=review_text)
# 3. Call the API
client = openai.Client()
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0
)
raw_content = response.choices[0].message.content
# 4. The "Hope and Pray" Parsing Step
try:
# We often need hacky cleanup code here just in case
cleaned_content = raw_content.strip().replace("```json", "").replace("```", "")
data = json.loads(cleaned_content)
return data["sentiment"], data["reason"]
except (json.JSONDecodeError, KeyError):
# This happens way too often
return "Error", "LLM failed to follow JSON instructions"
# Usage
s, r = analyze_review_manual("The visuals were stunning, but the story put me to sleep.")
print(f"Sentiment: {s}\nReason: {r}")
The prompt has to insist on “Return ONLY JSON” and constrain the sentiment values. The application then strips possible Markdown, calls json.loads, and handles missing keys. Improving accuracy means manually rewriting the English instructions in PROMPT_TEMPLATE.
With DSPy: declare inputs and outputs
With DSPy, you define the input and output structure and let the framework handle prompt creation and parsing.
import dspy
# Configure the LM once
lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)
# 1. Define the "Signature" (The Interface)
class MovieSentiment(dspy.Signature):
"""Classify the sentiment of a movie review and explain why."""
review_text = dspy.InputField()
sentiment = dspy.OutputField(desc="Positive or Negative")
reason = dspy.OutputField(desc="A short explanation")
# 2. Define the Module (The Strategy)
# ChainOfThought automatically adds "Let's think step by step" logic
analyzer = dspy.ChainOfThought(MovieSentiment)
# 3. Run it
result = analyzer(review_text="The visuals were stunning, but the story put me to sleep.")
# Access attributes directly (No parsing needed)
print(f"Sentiment: {result.sentiment}")
print(f"Reason: {result.reason}")
What the generated prompt looks like
The DSPy example has no handwritten prompt. DSPy generates one from the MovieSentiment signature.
For MovieSentiment with ChainOfThought, a simplified version looks like this:
Classify the sentiment of a movie review and explain why.
Review: {review_text}
Let's think step by step.
Sentiment: Positive or Negative
Reason: A short explanation
DSPy constructs it from five parts:
- The task description comes from the signature’s docstring (
"""Classify the sentiment of a movie review and explain why."""). - Each
InputFieldbecomes a labeled section; here,review_textsupplies the label. ChainOfThoughtadds “Let’s think step by step.” to encourage reasoning.- Each
OutputFielduses itsdescvalue, here"Positive or Negative"and"A short explanation", to guide the model. - DSPy adapts the structure to the selected model’s chat template.
The actual API prompt is more structured and includes system messages. The useful distinction is that these instructions are derived from the signature rather than maintained in a separate prompt string.
What DSPy changes
1. Modules replace prompt rewrites
Changing the strategy from a simple prediction to a “Reason-Act” loop that can search Wikipedia requires changing the module rather than rewriting the entire prompt:
analyzer = dspy.ReAct(MovieSentiment, tools=[...])
2. Model-specific formatting stays in the framework
Moving from GPT-4 to a locally hosted Llama-3 model often requires different prompts for different chat templates. DSPy handles that translation at runtime, so the application code usually stays unchanged.
3. Metrics can optimize the prompt
When the sentiment analyzer is not accurate enough, the manual approach is to rewrite the prompt and try different few-shot examples by intuition.
DSPy instead evaluates prompt variants against a dataset and a metric. Its optimizers tune instructions and examples in a process similar to hyperparameter tuning.
To use a DSPy optimizer, you need three things:
- A Dataset: A list of inputs and expected outputs (e.g., 20 examples of movie reviews and their correct labels).
- A Metric: A function that defines success (e.g.,
correct_sentimentwhich returnsTrueif the prediction matches the label). - An Optimizer: A strategy module. While standard optimizers like
BootstrapFewShotjust pick good examples, optimizers like GEPA (Agrawal et al., 2025) use LLM reflection and an evolutionary search to rewrite your prompt instructions for you.
The optimizer then “compiles” the program:
import dspy
# 1. Define the Metric
# GEPA's metric returns a score plus textual feedback. The feedback is what
# the reflection model reads to understand why a prediction went wrong.
def validate_sentiment(gold, pred, trace=None, pred_name=None, pred_trace=None):
correct = gold.sentiment.lower() == pred.sentiment.lower()
feedback = (
"Correct label."
if correct
else f"Expected '{gold.sentiment}' but got '{pred.sentiment}'."
)
return dspy.Prediction(score=float(correct), feedback=feedback)
# 2. Initialize the GEPA Optimizer
# GEPA (Genetic-Pareto) uses an LLM to reflect on failures and rewrite
# the instructions of each predictor in the program.
optimizer = dspy.GEPA(
metric=validate_sentiment,
auto="light", # Budget preset: "light", "medium" or "heavy"
reflection_lm=dspy.LM("openai/gpt-4o", temperature=1.0), # A strong model that proposes new instructions
)
# 3. Compile!
# trainset/valset are lists of dspy.Example(review_text=..., sentiment=...).with_inputs("review_text")
compiled_analyzer = optimizer.compile(analyzer, trainset=trainset, valset=valset)
What compile() does
Unlike simpler optimizers that search for examples, GEPA cycles through reflection and mutation:
- Evaluation: It runs the current prompts against your dataset and identifies where they fail (e.g., “The model failed to produce JSON” or “The model confused sarcasm with negativity”).
- Reflection: It uses a meta-model (an LLM) to look at the error traces and diagnose why the prompt failed.
- Mutation (Evolution): It “mutates” the prompt instructions to fix those specific errors. For example, it might rewrite
"Explain why"to"Extract a concise substring proving the sentiment"if the descriptions were too vague. - Selection: Instead of keeping only the single best prompt, it keeps a Pareto front: every candidate that is the best on at least one validation example survives. New mutations are drawn from that front, so ideas that solve different subsets of the data are not discarded too early.
The resulting compiled_analyzer has the same interface as the original object, but contains optimized instructions. You maintain only the signature; the optimizer writes the prompt text and keeps the version that scores best on your metric.