AI安全评估AI safety evaluation
独立检验模型安全性
每条都摘自下方编号的来源原文;只写事实,不写预测或建议。
技术简介
- Apollo此前在后训练完成、公开发布前评估前沿模型。1
Apollo Research’s previous evaluations of frontier models took place after post-training and shortly before public release.
已有应用
- METR将评估OpenAI迄今规模最大的强化学习训练中的未对齐行为,包括现有测试与监控可能遗漏的问题。2
* **Alignment assessment:** For the first phase, METR will assess misaligned behavior in our largest reinforcement learning run to date, including failures that our existing tests and monitors may have missed.
- OpenAI此前与METR开展过为期三周的监控红队测试,并开放代码库、监控器、相关轨迹及研究人员讨论。2
We recently conducted a 3 week long monitor red teaming exercise with METR, which granted them access to our codebase, monitors, relevant trajectories, and discussions with OpenAI research staff for the purpose of better understanding our monitors.
主要参与者
- OpenAI已与Apollo Research、METR、Transluce和Goodfire签订首阶段嵌入式评估协议。2
We have signed agreements with four independent assessors—Apollo Research, METR, Transluce, and Goodfire—for our first phase of embedded assessments.
未定事项
- Apollo试点中其他评估主张的详细访问安排仍待商定。1
The detailed access arrangements for the other claims in the assessment remain to be agreed.