2026-10-27 · America/Los_Angeles · 论文 · #6

S3AC: Stabilized, Selective, and Scalable Actor-Critic for LLM Reasoning

Critic-free methods are widely used in reinforcement learning with verifiable rewards (RLVR) for long chain-of-thought (CoT) reasoning, avoiding difficult and costly critic training. Common alternatives lack token-level credit assignment and typically rely on grouped rollouts to estimate prompt-specific baselines. A critic can address both limitations, but long-CoT critic training is hindered by insufficient coverage of trajectories with diverse lengths and early gradients dominated by the newly initialized value head. We introduce S3AC (Stabilized, Selective, and Scalable Actor-Critic) for…

热度 58.0 / 100；排名与评分保留该期记录。