Tiny Model,
Big Jobs.
Optimizing LLMs with DSPy + GEPA
But first — who am I?
Arun Prasath E G — 20+ years · co-founder, eight times over.
Sounds like an executive. I'm a builder.
Writing a prompt is easy.
Making it reliable isn't.
Where we're headed — five moves.
From hand-tuned guesswork to a prompt you compile. Here's the whole path — we'll walk it end to end.
Five parts — each one earns the next. By the end, you'll have the whole recipe.
Recognise this?
No metric. No data. Just vibes.
one score.
The test: a warehouse robot
Let's find out how good the hand-written way really is — with a number, not a vibe.
The brain doing it: a tiny 8B — small enough to live on the robot.
Sounds trivial. One score: did it finish — intact?
You already know a model.
Before the 8B: what is a model? Celsius in, Fahrenheit out — you've used this one since school. Now look at where the numbers live.
The temperature flows through. The 1.8 and the 32 stay. Those two numbers are the model.
Nobody told it 1.8.
Hide the constants. Hand it real thermometer readings instead. Then loop: guess, check, nudge. Watch.
Big error → big nudge. Tiny error → tiny nudge. The dials settle where the error dies. An LLM is this exact loop — with 8,000,000,000 dials.
8B? Eight billion what?
A language model is one giant equation with a single job: guess the next word. A parameter is one number inside it — one tiny dial.
| "the vase is ___" | fragile | heavy | blue | …every other word |
|---|---|---|---|---|
| the | −0.27 | −0.31 | −0.24 | … |
| vase | 0.83 | −0.52 | −1.87 | … |
| is | 0.10 | 0.12 | 0.12 | … |
| add the column ↓ | 0.66 | −0.71 | −1.99 | … |
| share of 100% | 71% | 18% | 5% | 6% |
Biggest total wins: fragile 0.66 → 71%. 8 billion = crossings between words — not words.
Where does 0.83 come from?
Nobody typed it. Every dial starts as a random number. Training then shows the model trillions of sentences and plays the same game every time: hide the next word, let it guess, reveal the answer. Wrong guess → every dial gets turned a tiny bit, in the direction that would have made the right answer more likely.
Sentences with vase and fragile together kept turning this dial up. After trillions of turns it stopped at 0.83. That's all "learned" means.
…and how do dials turn into fragile 71%?
the (−0.27) + vase 0.83 + is 0.10 = 0.66
heavy's column adds to −0.71 · blue's adds to −1.99
fragile 0.66 → 71% · heavy −0.71 → 18% · blue −1.99 → 5% · every other word 6%
(that last squash is called softmax)
All numbers made up so the math stays small. A real 8B does exactly this — with 8 billion dials, ~30 layers of adding-and-multiplying, and every word in the dictionary getting a score (we show three).
Why dials cost gigabytes
Eight billion learned dials — and every one has to sit in memory, loaded and ready, about two bytes each. Now just multiply.
So 8B isn't a compromise — it's the biggest brain the robot can carry.
Then why not a 1B? Or 100M?
Fair question — smaller is lighter and faster still. But dials are capacity: language, patterns, the ability to follow instructions. Shrink too far and the model loses the plot.
Now — how the world works
The robot moves between locations, and the world talks back. Note the wrap_station — the vase is fragile. A brain working a world like this has a name: an agent.
But what is an agent?
One vase, four zones, four actions. But this is not one question with one answer — it's a goal that takes many steps, in a world that reacts to every move.
Same model. The difference is the loop — and a goal it keeps working toward.
The agent loop — see, decide, act.
Every step, our robot runs the same three-beat cycle — and only one beat is a prompt.
The brain — how it decides — is just a prompt. Make that reliable, and the whole agent gets reliable.
The hand-written prompt
That decide step? Today it's just this — a wall of hand-written rules. Start simple; each new failure adds a line.
Now run your wall of rules.
All 20 tasks, logged. And it still breaks — on the one rule you never thought to write.
"Just use a bigger model?"
The obvious fix. So we tried it — 30× more parameters, the exact same prompt.
Bigger bought 15 points — at 4× the price. The words are still the problem. So let's fix the words.
Programming, not prompting.
There's a framework built for exactly this. Let's build it — piece by piece. →
What is DSPy?
Declarative Self-improving Python — from Stanford. A framework for programming language models, not prompting them.
Three pieces, in order: Signature → Module → Optimizer. We'll build them one at a time.
Signature — declare the task
Not a prompt — a typed spec: what goes in, what comes out. DSPy turns it into the prompt.
Module — the strategy
A module wraps a signature and decides how to get the answer. Swap the module, change how it thinks.
We'll look at the first three — then pick one for our robot.
Predict — just answer
The simplest module: inputs in, output out — no reasoning, no tools.
Straight to the answer — no working shown, and nothing to catch a wrong move.
ChainOfThought — think, then answer
Same signature. DSPy quietly adds a reasoning field — the model works it out before it answers.
That new reasoning line is the whole difference — it thinks before it acts, so it's far more reliable on anything with steps.
ReAct — reason, act, repeat
An agent: it reasons, calls a tool, reads the result, and loops until the job is done.
Remember the agent loop — see, decide, act? ReAct is exactly that, built into one module. (We keep ours explicit, so it's clear what the optimizer tunes.)
Now tell it why it failed
Optimizing needs a score. The best optimizers want more — the reason behind it.
Optimizer — improve it from data
Hand it your module + a metric + a few examples. It rewrites the prompt to score higher — you never touch the words.
What is GEPA? — a learning loop.
Genetic-Pareto — it evolves prompts from their own failures, automatically. From the 2025 paper "Reflective Prompt Evolution Can Outperform Reinforcement Learning."
40% → 95%, one loop at a time.
What GEPA wrote — that you didn't.
Why keep a 40% prompt? Watch the record board.
GEPA keeps a score for every candidate on every task — the 20 val tasks, side by side.
Task #2 — the champion's only ✗. Candidate 0 holds the only ✓ → unbeaten there → it stays. That's Pareto. "Put blue_box, red_box and green_box into bin_1"
Staying = quizzed again — and its failures write candidate 2 · 95%. That's Genetic.
Winners stop failing. Losers keep teaching. One prompt ships.
Same task. Now it works.
The prompt did all the work.
Bigger bought 15 points (45%). Better words bought 65 points (95%) — on the same tiny 8B, at a fraction of the cost, small enough to ride the robot. You never wrote the winning prompt.
measured on 20 held-out tasks
New prompt.
Same dials. Different light.
Remember the box of dials? They've been frozen since training — GEPA never touched one. The only thing that changed is which ones fire.
Fine-tuning turns the dials. Prompting chooses which ones fire. GEPA automates the search for the words that light the right ones.
It's still just a prompt, right?
Right — but remember the treadmill: every rule you hand-bolt can silently break the one before. Developers fixed this long ago — Test-Driven Development.
Write it before the code — it fails. "Working" now has a definition.
Write code until the test goes green.
New requirement? Add its test, build on top — the old tests prove nothing broke.
A requirement = tasks + a metric. "Handle fragile items" → 20 tasks. Hand prompt: 30% — a fact, not a feeling.
GEPA writes the words until the metric climbs — 95%.
New requirement? Add its tasks, recompile — the old 20 prove the vase still survives.
Every requirement becomes a test. Every change ships with proof. New rule, new model, new task — recompile, and know.
The robot was just the demo.
The method works anywhere an LLM has a job — and you can score it.
If you can score it, GEPA can optimize it.
The whole recipe — on one slide.
Everything we just built, in four moves. This is DSPy + GEPA, end to end.
You declare the intent and a way to score it. The optimizer writes the words — and keeps only the ones that win.