快讯
Fuli Luo回应ValsAI:mimoagent计划收紧运行时反作弊脚本以规避重建镜像
Fuli Luo回应ValsAI,介绍其内部RL环境采用"可作弊环境+独立防作弊预处理脚本"的拆分设计:新作弊模式出现时只需更新脚本,代价是须搭配该脚本使用;团队计划收紧mimoagent中的运行时反作弊脚本以免重建镜像。转发者Adithya S K称其Harbor实现的Mimo RL Envs一周前也遇到同类奖励作弊问题,已于3天前修复大部分。
事件来源
查看原文
RT Fuli Luo Re @ValsAI Nice work. Internally, our RL setup is hackable environments plus a separate anti-hack preprocessing script. The upside of that split is that when a new hack pattern shows up, we only update the script and do not rebuild the environments. The downside is that what we ship is not out-of-the-box: it has to be paired with the preprocessing script. After you flagged this, our plan was to tighten the on-the-fly anti-hack scripts in mimoagent so we would not have to rebuild images. That said, @adithya_s_k's approach of cleaning the leaks and then rebuilding the images is more convenient for people running other harnesses. https://x.com/adithya_s_k/status/2108269775336190129 Adithya S K: We saw many of the same reward hacking issues when we tested these environments a week ago with our Harbor implementation of Mimo RL Envs, and patched most of them 3 days ago. Please try breaking them and let us know what you find. We want to be fully transparent and share what