RL post-training turns a 4B Qwen model into 1.81x faster Postgres query plans
- A 4B open-weights Qwen model post-trained with supervised fine-tuning and agentic RL produced Postgres query plans with a 1.81x geometric mean speedup and a 44.7% summed latency drop across 113 join-heavy queries, beating the planner's default plans.
- The same model started out unable to produce any valid plan for 99 of those 113 queries, so most of the gain came from teaching it the hint syntax before it could be scored on latency.
- Plans reach Postgres as pg_hint_plan optimizer hints (HashJoin, Leading, NestLoop, IndexScan), and each rollout's reward is the execution time Postgres reports back, fed into a custom GRPO variant built for a noisy measurement environment.
- The run cost roughly $800 for about 95 hours on a rented 2x H100 SXM node from Lambda plus $400 in OpenAI API fees for off-policy distillation over 500 GPT-6 Astra agent trajectories, with vLLM and the trainer on the rented node and four Postgres containers on the author's desk.
- The benchmark is favorable: an 8GB IMDb dataset that fits in memory, shared_buffers capped to a fraction of that, queries warmed before measurement, and read-only SELECTs, which commenters note makes the speedup hard to extrapolate to scale or OLTP.
Hacker News opinions
How do you know the hinted plan actually does what the query asked for? The model could hand back something that isn't the same query at all.
...make no mistakes :)
pg_hint_plan has a debug log that shows whether Postgres actually used the hint or ignored it. I leaned on that during evaluations.
I'd assume pg_hint_plan only accepts hints that form a valid plan for the query. Generation is the expensive, heuristic part and verification is cheap, which is exactly why an LLM fits here.
Optimizers aren't deterministic. Plan choice depends on summary statistics about the data that can be stale, so a plan can flip on you for no obvious reason.
I noticed the $800 of H100 rental and $400 of API fees aren't in the benchmark numbers. Even so, can you afford to spend hours retraining a 4B model every so often just to keep it picking good plans?
That's comparable to a scheduled backup, so an acceptable maintenance window. Compilers have been hand-crafted for decades and I don't think LLMs add much architecturally there.
Admitting that frontier intelligence is extremely powerful and that the distillation came off Astra trajectories invites accusations of distillation between closed and open models.
Small models distilling from big ones isn't really the fight. The fight is frontier lab against frontier lab on big models.
Frontier labs trained on the entire internet, copyrighted material included, so they have a hard time claiming moral high ground about distillation.
Why is this write-up so long? It'd take me five days just to get through it.
Treat it like a paper, nobody asks why a paper is long. And you can just have an AI summarize it and answer questions about the parts you care about.
81% faster on an 8GB dataset that fits entirely in memory, shared_buffers constrained below that, queries warmed and read-only SELECTs. Hard to say those plans really beat Postgres heuristics at scale or on real OLTP.
A 4B model that probably ran through 8GB of RAM several times to produce one plan. At some point we should be talking about CUDA-accelerating Postgres instead.
Optimal plan construction is math-heavy and varies by workload. Add just-in-time indexes and the space grows even faster. An LLM is a blunt weapon here, I want an AlphaGo-style neural heuristic instead.