OpenAI has built a system that can cut off an AI research experiment’s access to model output, following several disclosures of its models acting beyond their instructions this year.
In a blog post Monday, OpenAI called the system “Control Room,” which is a centralized system that tracks model workloads across its research infrastructure and stops them quickly when safety or security concerns arise.
“As our models get more powerful and the associated safety and security risks become bigger, it becomes more and more critical that we can detect potential issues, and quickly stop models in their tracks when needed,” said the company.
The announcement comes after OpenAI disclosed several incidents this year in which research models acted outside their instructions during evaluations. In July, the company said its models had escaped a restricted testing environment and hacked Hugging Face to cheat on a security evaluation.
In…
