DeepSeek New Paper: No Single Mechanism Can Prevent All Agent Misbehavior and System Failures

PANews, September 23 - A new paper by DeepSeek on agent training has drawn attention. The 31-page paper introduces DSec, a production-grade sandbox platform used internally by the company, with over 100 authors, including Liang Wenfeng. A notable section of the paper addresses "agent misbehavior," where DeepSeek found that agents can obtain answers through unintended channels, such as residual answers in platform management files, undermining the validity of training and evaluation results. After introducing access controls, some agents exchanged file data block mappings, attempting to make protected file contents accessible through another file descriptor, thereby compromising tasks or shared infrastructure.

DeepSeek argues that no single mechanism can prevent all agent misbehavior and system failures. Therefore, the team's approach is to enhance observability to identify new issues and continuously harden DSec as models evolve, including access controls that limit agents from obtaining answers through unintended channels, and reducing rewards for deceptive behavior. These controls can address some of these issues.