How “reverse game theory” could keep AI agents in check

A new approach known as “reverse game theory” aims to control powerful AI models by designing systems that incentivize them to stick to the rules. As AI models become more capable, safety researchers are scrambling to ensure they are “aligned” with human desires—to make sure, in other words, that rogue agents don’t break the law,…

面對越來越強的 AI 模型，一些研究者主張不再試圖看懂模型內部，而是用機制設計讓模型在激勵下守規矩。文章介紹了讓模型互相預測、互相審計等具體方案，也指出這些方案依賴“智能體理性”這一未必成立的假設。