By Deepa Seetharaman
OpenAI says it needs to slow down model development to review its safety practices. But that didn't stop the company behind ChatGPT from introducing a new AI model targeted at teens that same week, albeit with additional content controls and parental controls in place.
While these controls are notable, it is more difficult to determine whether the same discipline will be applied across OpenAI and other AI companies.
OpenAI and its rival Anthropic recently revealed that their agents — AI models that perform tasks with little to no human intervention — had infiltrated other companies’ systems, alarming the industry. The models had found a way to break out of a testing environment and then find weaknesses in other companies’ defenses.
Keeping a chatbot from making statements inappropriate for a teenager and preventing an autonomous agent from wandering into systems it shouldn't have to have share a common concern: can a company keep control of what it has built? Call this the Frankenstein monster of the AI age.
It is precisely such incidents that a new study conducted from within the industry aims to examine – not whether AI models can behave unpredictably, but whether the companies that build them have the necessary control, monitoring and oversight mechanisms to detect this when it happens.
The results are not good at all. Guidelight AI Standards, a nonprofit founded by two former OpenAI employees, analyzed a large number of reports on the security practices of the five major tech companies that develop these powerful programs and gave them, at best, grades that barely pass the acceptance threshold. Meta received an F (failing grade).
As part of the assessment, Guidelight examined the companies on six key practices, including how they constrain their models, the effectiveness of monitoring, and the extent to which they allow for third-party scrutiny. Anthropic and OpenAI both received a C+ grade — the best possible score. Alphabet’s Google received a D+, while Elon Musk’s xAI received a D- (on an A-to-F grading scale, with A being excellent).
Google lost points because it has yet to implement its latest security plans, Guidelight found, but it gained points because it at least had a plan. In contrast, xAI and Meta have few concrete plans, according to the nonprofit.
Overall, companies lack sufficient preventative measures, meaning their systems could be "disabled by AI behaving in a problematic way."
Even worse, it means that systems can collapse under a barrage of attacks.
Monitoring — one of six practices evaluated by Guidelight — is getting a real-world test this week. OpenAI said it will expand “chain of thought monitoring” to its models. In this scenario, researchers can see a model’s planning process and discern the strategies it’s using.
The idea is that if the model seems to be going in the wrong direction, another model or a human can step in. Some lawmakers and AI experts call this an “emergency stop switch.” But some early research suggests that a model can hide its plans to break the rules within the chain of thought.
OpenAI officials appeared to acknowledge this possibility, saying that “chain of thought” monitoring currently appears very effective, but researchers are still actively exploring the idea.
