Rising alarm about AI’s perils is thrusting into the spotlight a collection of little-known research groups focused on the technology’s safety—and fueling questions about their ability to effectively monitor the industry’s biggest companies.

Both Anthropic Chief Executive Dario Amodei and OpenAI CEO Sam Altman promised in recent days to embed evaluators from such outside research organizations inside their companies to help ensure they develop AI at a safe pace. Other executives including Microsoft CEO Satya Nadella also endorsed the idea of embedding independent evaluators in companies.

A team of cybersecurity researchers with startup Hacktron AI used Anthropic’s Claude to hack into OpenAI, gaining access to a key repository of OpenAI’s software, the researchers revealed Thursday.

The hack, which occurred on July 25, further demonstrates AI companies’ vulnerability to ...