RL post-training mostly fixes problems the model already half-solves, and hard problems with pass@32=0 stay unsolved, a bias the author calls the Matthew Effect
- Splitting the AIME 2025 eval into difficulty buckets during Olmo 3.1 RL-Zero Math training shows the gains are concentrated in easy problems: questions the pre-RL model scored 0 on at pass@32 mostly end training at pass@32=0, while easy questions go from somewhat solved to mostly solved.
- Michael Noukhovitch names this the Matthew Effect in RL for LLMs: RL improves performance on a task in proportion to the model's initial competence, making easy tasks easier while hard tasks stay difficult.
- The same difficulty bias appears outside math, in code RL on DeepCoder (LCBv5) and agentic coding on DeepSWE (SWEBench) split by task length, where RL gains scale with how easy the problems are.
- The cause is not just GRPO signal loss from zero-reward groups: on GSM8k Platinum, training Qwen 2.5 0.5B Instruct with 64 prompts and k=4 completions solved hard problems better than 8 prompts with k=32.
- The author proposes a method called Never Give Up (NGU) to counter the effect, with the paper on arXiv and code on GitHub.