Sam Altman identifies two biggest risks facing AI's future
The OpenAI CEO said losing control of AI alignment or concentrating too much power in one lab or country must be avoided
AI expert on alarm bells sounding across tech industry
ControlAI US Executive Director Connor Leahy discusses the warning signals the artificial intelligence industry has been showing us and AIs scary possibilities on The Claman Countdown.
OpenAI CEO Sam Altman outlined two scenarios that he sees could derail progress in developing artificial intelligence (AI) and must be avoided.
Altman wrote in a Sunday night post on X that the two areas of concern involve the loss of control over AI's alignment, or the concentration of too much power by a country or AI lab.
"First we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities," Altman wrote.
"Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian," he added.
MICROSOFT UNVEILS CODE OF CONDUCT FOR AI MODELS AS SAFETY CONCERNS MOUNT
OpenAI CEO Sam Altman said that AI could overtake human control of the future if not properly managed. (Justin Sullivan/Getty Images)
"Avoiding these two threats requires walking a narrow middle path; for example, one country could gain too much power. Another example is one lab ending up with too much power," Altman wrote.
The OpenAI CEO's post served as a follow-up to an earlier post that said U.S. companies developing AI need to adhere to safety standards, adding that he and the company would "welcome a federal framework that sets consistent safety requirements for frontier AI."
Altman said that while companies have previously created responsible scaling policies and preparedness frameworks, the current state of the AI field requires a new approach.
SAM ALTMAN SAYS OPENAI WON'T GO PUBLIC IN 2026 AMID AI SAFETY CONCERNS
OpenAI CEO Sam Altman warned that AI could lead to the concentration of power with one country or one AI lab. (Jason Redmond/Getty Images)
He said that, for example, OpenAI now goes through a process to "formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases."
Altman's comments come as he and other AI leaders at U.S. companies discuss ways to ensure the alignment and safety of AI models as they become more sophisticated, with researchers warning there is a chance an AI superintelligence could wipe out humanity within a decade.
Anthropic CEO Dario Amodei called for AI companies to pace the development of frontier models. (Anna Moneymaker/Getty Images)
Anthropic CEO Dario Amodei wrote an essay calling for AI labs to "pace the frontier" of model development, including through the use of third-party evaluators who have employee-level access to company systems and can verify adherence to safety measures, report on incidents and assess models' alignment during training.
Altman said in a post on X that he agrees with that approach, adding that "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."