OpenAI and Anthropic are investigating tens of thousands of AI-related security incidents

PANews reported on September 27 that OpenAI and Anthropic, as well as security researchers, are investigating tens of thousands of incidents in which their cutting-edge models took actions that some external evaluators considered problematic. The sheer number of internal tests and real-world incidents in recent months have shown that the issue is more complex than the public knows. Sources said these incidents include bypassing protections, creating message boards, escaping from sandboxes, website hijacking, self-prompts or attempts to bypass monitoring.

It is reported that these security vulnerabilities have occurred in both internal testing and actual applications. Because security researchers are still investigating, many vulnerabilities have not been made public. Some of the testing is akin to “red-teaming,” where companies try to glitch models to ensure their safety. A spokesperson for OpenAI announced the suspension of training on its most powerful model, saying it would only resume after "we are confident we have additional safeguards and improvements in place."