Conservative Meta-Agent Search: Architecture Adaptation under Non-Stationarity
Resumo indisponível. Consulte o paper original.
PAPER / ARXIV:2610.15840
Kaijie Zhu, Weixiang Yan, Omar Attia, Abbas Kazerouni, Alexander Toshev, Andrew Szot
RESUMO
Reinforcement learning (RL) for large language model (LLM) agents remains challenging in long-horizon tasks with sparse outcome rewards, where assigning credit to individual actions is difficult. We introduce LLM Value Function RL (LVRL), which uses the pretrained knowledge and in-context learning ability of an LLM to estimate policy-aware state values without finetuning a separate critic, conditioning the value model on group context and calibrating predictions using known values at initial and terminal states. We further extend this through on-policy distillation (LVOPD), focusing teacher supervision on actions associated with decreases in estimated value. Across AppWorld, BabyAI, and SWE-bench Verified, LVRL consistently outperforms GRPO and PPO, with absolute improvements up to 13.6 and 8.6 percentage points over GRPO, and bridges BabyAI from total failure to 75.0% success.
NO MESMO MAPA
Resumo indisponível. Consulte o paper original.
Resumo indisponível. Consulte o paper original.
Resumo indisponível. Consulte o paper original.
Resumo indisponível. Consulte o paper original.
Resumo indisponível. Consulte o paper original.