How “reverse game theory” could keep AI agents in check

A new approach known as “reverse game theory” aims to control powerful AI models by designing systems that incentivize them to stick to the rules. As AI models become more capable, safety researchers are scrambling to ensure they are “aligned” with human desires—to make sure, in other words, that rogue agents don’t break the law,…

面对越来越强的 AI 模型，一些研究者主张不再试图看懂模型内部，而是用机制设计让模型在激励下守规矩。文章介绍了让模型互相预测、互相审计等具体方案，也指出这些方案依赖“智能体理性”这一未必成立的假设。