Stop Prompting, Start Programming with DSPy

After reading about DSPy in Pedram Navid’s excellent blog post, I wanted a smaller example that showed what the framework changes in practice.

The example takes a movie review, classifies it as Positive or Negative, and gives a short reason.

Without DSPy: prompt and parser stay coupled

The manual version couples application logic to a prompt string. It also handles the API call, formatting instructions, cleanup, and JSON parsing itself.

# --- THE OLD WAY ---
import openai
import json

# 1. The Brittle Prompt String
# If you change a word here, you might break the JSON parser later.
PROMPT_TEMPLATE = """
Analyze the sentiment of the following movie review.
Return ONLY a JSON object with keys: "sentiment" and "reason".
The sentiment must be exactly "Positive" or "Negative".
Do not include markdown formatting like ```json at the start or end.

Review: "{review_text}"
"""

def analyze_review_manual(review_text):
    # 2. Manually construct the prompt
    prompt = PROMPT_TEMPLATE.format(review_text=review_text)

    # 3. Call the API
    client = openai.Client()
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        temperature=0
    )

    raw_content = response.choices[0].message.content
    
    # 4. The "Hope and Pray" Parsing Step
    try:
        # We often need hacky cleanup code here just in case
        cleaned_content = raw_content.strip().replace("```json", "").replace("```", "")
        data = json.loads(cleaned_content)
        return data["sentiment"], data["reason"]
    except (json.JSONDecodeError, KeyError):
        # This happens way too often
        return "Error", "LLM failed to follow JSON instructions"

# Usage
s, r = analyze_review_manual("The visuals were stunning, but the story put me to sleep.")
print(f"Sentiment: {s}\nReason: {r}")

The prompt has to insist on “Return ONLY JSON” and constrain the sentiment values. The application then strips possible Markdown, calls json.loads, and handles missing keys. Improving accuracy means manually rewriting the English instructions in PROMPT_TEMPLATE.

With DSPy: declare the contract

With DSPy, you define the input and output structure and let the framework handle prompt creation and parsing.

import dspy

# Configure the LM once
lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)

# 1. Define the "Signature" (The Interface)
class MovieSentiment(dspy.Signature):
    """Classify the sentiment of a movie review and explain why."""
    review_text = dspy.InputField()
    sentiment = dspy.OutputField(desc="Positive or Negative")
    reason = dspy.OutputField(desc="A short explanation")

# 2. Define the Module (The Strategy)
# ChainOfThought automatically adds "Let's think step by step" logic
analyzer = dspy.ChainOfThought(MovieSentiment)

# 3. Run it
result = analyzer(review_text="The visuals were stunning, but the story put me to sleep.")

# Access attributes directly (No parsing needed)
print(f"Sentiment: {result.sentiment}")
print(f"Reason: {result.reason}")

What the generated prompt looks like

The DSPy example has no handwritten prompt. DSPy generates one from the MovieSentiment signature.

For MovieSentiment with ChainOfThought, a simplified version looks like this:

Classify the sentiment of a movie review and explain why.

Review: {review_text}

Let's think step by step.

Sentiment: Positive or Negative
Reason: A short explanation

DSPy constructs it from five parts:

The task description comes from the signature’s docstring ("""Classify the sentiment of a movie review and explain why.""").
Each InputField becomes a labeled section; here, review_text supplies the label.
ChainOfThought adds “Let’s think step by step.” to encourage reasoning.
Each OutputField uses its desc value, here "Positive or Negative" and "A short explanation", to guide the model.
DSPy adapts the structure to the selected model’s chat template.

The actual API prompt is more structured and includes system messages. The useful distinction is that these instructions are derived from the signature rather than maintained in a separate prompt string.

What DSPy changes

1. Modules replace prompt rewrites

Changing the strategy from a simple prediction to a “Reason-Act” loop that can search Wikipedia requires changing the module rather than rewriting the entire prompt:

analyzer = dspy.ReAct(MovieSentiment, tools=[...])

2. Model-specific formatting stays in the framework

Moving from GPT-4 to a locally hosted Llama-3 model often requires different prompts for different chat templates. DSPy handles that translation at runtime, so the application code usually stays unchanged.

3. Metrics can optimize the prompt

When the sentiment analyzer is not accurate enough, the manual approach is to rewrite the prompt and try different few-shot examples by intuition.

DSPy instead evaluates prompt variants against a dataset and a metric. Its optimizers tune instructions and examples in a process similar to hyperparameter tuning.

To use a DSPy optimizer, you need three things:

A Dataset: A list of inputs and expected outputs (e.g., 20 examples of movie reviews and their correct labels).
A Metric: A function that defines success (e.g., correct_sentiment which returns True if the prediction matches the label).
An Optimizer: A strategy module. While standard optimizers like BootstrapFewShot just pick good examples, optimizers like GEPA use evolutionary algorithms to rewrite your prompt instructions for you.

The optimizer then “compiles” the program:

from dspy.teleprompt import GEPA

# 1. Define the Metric
# This tells the optimizer what "success" looks like
def validate_sentiment(example, prediction, trace=None):
    # We check if the sentiment matches exactly
    return example.sentiment == prediction.sentiment

# 2. Initialize the GEPA Optimizer
# GEPA (Genetic Evolutionary Prompt Optimization) is an advanced strategy 
# that evolves both the instructions and the few-shot examples.
optimizer = GEPA(
    metric=validate_sentiment,
    num_generations=5,       # How many "evolutionary steps" to run
    population_size=10,      # How many prompt variations to keep in the pool
    mutation_rate=0.5,       # How aggressively to rewrite the instructions
    verbose=True
)

# 3. Compile!
# DSPy now acts as an automated prompt engineer, evolving your program
compiled_analyzer = optimizer.compile(student=analyzer, trainset=my_dataset)

What `compile()` does

Unlike simpler optimizers that search for examples, GEPA cycles through reflection and mutation:

Evaluation: It runs the current prompts against your dataset and identifies where they fail (e.g., “The model failed to produce JSON” or “The model confused sarcasm with negativity”).
Reflection: It uses a meta-model (an LLM) to look at the error traces and diagnose why the prompt failed.
Mutation (Evolution): It genetically “mutates” the prompt instructions to fix those specific errors. For example, it might rewrite "Explain why" to "Extract a concise substring proving the sentiment" if the descriptions were too vague.
Selection: It keeps the best performing prompts (the “fittest”) and discards the rest, repeating this over several generations.

The resulting compiled_analyzer has the same interface as the original object, but contains optimized instructions and few-shot examples. The contract stays in code; the prompt becomes an artifact produced against the metric.

AI Chat