Skip to main content
AI
View as Markdown Suggest changes

Stop Prompting, Start Programming with DSPy

· Reading time: 3 min
Stop Prompting, Start Programming with DSPy

After reading about DSPy in Pedram Navid’s excellent blog post, I wanted a smaller example that showed what the framework changes in practice.

The example takes a movie review, classifies it as Positive or Negative, and gives a short reason.

Without DSPy: prompt and parser stay coupled

The manual version couples application logic to a prompt string. It also handles the API call, formatting instructions, cleanup, and JSON parsing itself.

# --- THE OLD WAY ---
import openai
import json

# 1. The Brittle Prompt String
# If you change a word here, you might break the JSON parser later.
PROMPT_TEMPLATE = """
Analyze the sentiment of the following movie review.
Return ONLY a JSON object with keys: "sentiment" and "reason".
The sentiment must be exactly "Positive" or "Negative".
Do not include markdown formatting like ```json at the start or end.

Review: "{review_text}"
"""

def analyze_review_manual(review_text):
    # 2. Manually construct the prompt
    prompt = PROMPT_TEMPLATE.format(review_text=review_text)

    # 3. Call the API
    client = openai.Client()
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        temperature=0
    )

    raw_content = response.choices[0].message.content
    
    # 4. The "Hope and Pray" Parsing Step
    try:
        # We often need hacky cleanup code here just in case
        cleaned_content = raw_content.strip().replace("```json", "").replace("```", "")
        data = json.loads(cleaned_content)
        return data["sentiment"], data["reason"]
    except (json.JSONDecodeError, KeyError):
        # This happens way too often
        return "Error", "LLM failed to follow JSON instructions"

# Usage
s, r = analyze_review_manual("The visuals were stunning, but the story put me to sleep.")
print(f"Sentiment: {s}\nReason: {r}")

The prompt has to insist on “Return ONLY JSON” and constrain the sentiment values. The application then strips possible Markdown, calls json.loads, and handles missing keys. Improving accuracy means manually rewriting the English instructions in PROMPT_TEMPLATE.

With DSPy: declare inputs and outputs

With DSPy, you define the input and output structure and let the framework handle prompt creation and parsing.

import dspy

# Configure the LM once
lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)

# 1. Define the "Signature" (The Interface)
class MovieSentiment(dspy.Signature):
    """Classify the sentiment of a movie review and explain why."""
    review_text = dspy.InputField()
    sentiment = dspy.OutputField(desc="Positive or Negative")
    reason = dspy.OutputField(desc="A short explanation")

# 2. Define the Module (The Strategy)
# ChainOfThought automatically adds "Let's think step by step" logic
analyzer = dspy.ChainOfThought(MovieSentiment)

# 3. Run it
result = analyzer(review_text="The visuals were stunning, but the story put me to sleep.")

# Access attributes directly (No parsing needed)
print(f"Sentiment: {result.sentiment}")
print(f"Reason: {result.reason}")

What the generated prompt looks like

The DSPy example has no handwritten prompt. DSPy generates one from the MovieSentiment signature.

For MovieSentiment with ChainOfThought, a simplified version looks like this:

Classify the sentiment of a movie review and explain why.

Review: {review_text}

Let's think step by step.

Sentiment: Positive or Negative
Reason: A short explanation

DSPy constructs it from five parts:

  1. The task description comes from the signature’s docstring ("""Classify the sentiment of a movie review and explain why.""").
  2. Each InputField becomes a labeled section; here, review_text supplies the label.
  3. ChainOfThought adds “Let’s think step by step.” to encourage reasoning.
  4. Each OutputField uses its desc value, here "Positive or Negative" and "A short explanation", to guide the model.
  5. DSPy adapts the structure to the selected model’s chat template.

The actual API prompt is more structured and includes system messages. The useful distinction is that these instructions are derived from the signature rather than maintained in a separate prompt string.

What DSPy changes

1. Modules replace prompt rewrites

Changing the strategy from a simple prediction to a “Reason-Act” loop that can search Wikipedia requires changing the module rather than rewriting the entire prompt:

analyzer = dspy.ReAct(MovieSentiment, tools=[...])

2. Model-specific formatting stays in the framework

Moving from GPT-4 to a locally hosted Llama-3 model often requires different prompts for different chat templates. DSPy handles that translation at runtime, so the application code usually stays unchanged.

3. Metrics can optimize the prompt

When the sentiment analyzer is not accurate enough, the manual approach is to rewrite the prompt and try different few-shot examples by intuition.

DSPy instead evaluates prompt variants against a dataset and a metric. Its optimizers tune instructions and examples in a process similar to hyperparameter tuning.

To use a DSPy optimizer, you need three things:

  • A Dataset: A list of inputs and expected outputs (e.g., 20 examples of movie reviews and their correct labels).
  • A Metric: A function that defines success (e.g., correct_sentiment which returns True if the prediction matches the label).
  • An Optimizer: A strategy module. While standard optimizers like BootstrapFewShot just pick good examples, optimizers like GEPA (Agrawal et al., 2025) use LLM reflection and an evolutionary search to rewrite your prompt instructions for you.

The optimizer then “compiles” the program:

import dspy

# 1. Define the Metric
# GEPA's metric returns a score plus textual feedback. The feedback is what
# the reflection model reads to understand why a prediction went wrong.
def validate_sentiment(gold, pred, trace=None, pred_name=None, pred_trace=None):
    correct = gold.sentiment.lower() == pred.sentiment.lower()
    feedback = (
        "Correct label."
        if correct
        else f"Expected '{gold.sentiment}' but got '{pred.sentiment}'."
    )
    return dspy.Prediction(score=float(correct), feedback=feedback)

# 2. Initialize the GEPA Optimizer
# GEPA (Genetic-Pareto) uses an LLM to reflect on failures and rewrite
# the instructions of each predictor in the program.
optimizer = dspy.GEPA(
    metric=validate_sentiment,
    auto="light",                                   # Budget preset: "light", "medium" or "heavy"
    reflection_lm=dspy.LM("openai/gpt-4o", temperature=1.0),  # A strong model that proposes new instructions
)

# 3. Compile!
# trainset/valset are lists of dspy.Example(review_text=..., sentiment=...).with_inputs("review_text")
compiled_analyzer = optimizer.compile(analyzer, trainset=trainset, valset=valset)

What compile() does

Unlike simpler optimizers that search for examples, GEPA cycles through reflection and mutation:

  1. Evaluation: It runs the current prompts against your dataset and identifies where they fail (e.g., “The model failed to produce JSON” or “The model confused sarcasm with negativity”).
  2. Reflection: It uses a meta-model (an LLM) to look at the error traces and diagnose why the prompt failed.
  3. Mutation (Evolution): It “mutates” the prompt instructions to fix those specific errors. For example, it might rewrite "Explain why" to "Extract a concise substring proving the sentiment" if the descriptions were too vague.
  4. Selection: Instead of keeping only the single best prompt, it keeps a Pareto front: every candidate that is the best on at least one validation example survives. New mutations are drawn from that front, so ideas that solve different subsets of the data are not discarded too early.

The resulting compiled_analyzer has the same interface as the original object, but contains optimized instructions. You maintain only the signature; the optimizer writes the prompt text and keeps the version that scores best on your metric.

AI Chat

Messages you send are processed by the Google Gemini API to generate responses. Do not share sensitive personal data. See the privacy policy for details.