Group Prioritized Off-Policy Optimization (POPO

RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning

arXiv:2606.01281v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs). However, its effectiveness is substantially hindered by the prevalence of ineffective training data: many sampled prompts yield response groups that are either entirely correct or entirely incorrect, resulting in zero-variance rewards and limited learning signals. Recent...