Gradient-Guided Reward Optimization

Gradient-Guided Reward Optimization for Inference-time Alignment

arXiv:2606.09635v1 Announce Type: new Abstract: Ensuring the reliability of Large Language Models (LLMs) under distribution drift requires inference-time adaptation. While inference-time alignment methods such as Best-of-$N$ and rejection sampling are widely used, they frame the task as a sampling-intensive, reward-guided search, leading to two key limitations: their performance is bounded by the base model's generation quality, and their reliance on imperfect reward models makes them...