Put a Leash on AI, Not Pull Its Teeth — On Agent Safety: Capabilities Can Be Open, but Actions Must Be Bounded
Introduction In July, OpenAI and Hugging Face successively disclosed a rare agent safety incident. In an internal test of cyber capabilities, OpenAI’s model, in order to complete the evaluation objective, discovered and exploited a software vulnerability, broke through the originally restricted testing environment, obtained internet access, and then entered Hugging Face’s production infrastructure. OpenAI called it an “unprecedented” cybersecurity incident; Hugging Face, for its part, said the attack process involved tens of thousands of automated actions, and that its post hoc analysis logs recorded more than 17,000 events. This kind of thing is very easy to package as a science-fiction story about “AI developing self-awareness and trying to escape.” But…
Read more
