New venture by OpenAI co-founder aims to make superintelligent AI safe
Ensuring tomorrow’s AI is safe and trustworthy

Ilya Sutskever, a co-founder of OpenAI, has launched Safe Superintelligence Inc. (SSI), a company dedicated to addressing a critical issue in technology: the creation and control of superintelligent AI. SSI’s mission, as outlined on its website, is to solve “the most important technical problem of our time” – ensuring that superintelligent AI systems are safe.
Sutskever is joined by OpenAI engineer Daniel Levy and former Y Combinator partner Daniel Gross. Together, they aim to make safety a priority in AI development, alongside capability. Sutskever has long been concerned about the potential benefits and risks of superintelligent AI. In a 2023 OpenAI blog post co-authored with Jan Leike, he discussed the challenges of controlling AI systems that surpass human intelligence. The post highlighted the limitations of current alignment techniques, such as reinforcement learning from human feedback, which depend on direct human supervision.

“Humans won’t be able to reliably supervise AI systems much smarter than us, and so our current alignment techniques will not scale to superintelligence,” the blog stated. “We need new scientific and technical breakthroughs.”
Sutskever’s departure from OpenAI to establish SSI signals his desire to focus solely on this issue. SSI’s website emphasizes its singular focus on safe superintelligent AI, free from the distractions of management overhead or product cycles. The company aims to attract top engineers and researchers to concentrate exclusively on this mission.
While details on SSI’s specific strategies are sparse, Sutskever has shared some insights in an interview with Bloomberg. He mentioned that SSI plans to integrate safety protocols within AI systems during development, rather than adding safeguards afterward.
“By safe, we mean safe like nuclear safety as opposed to safe as in ‘trust and safety,’” Sutskever explained to Bloomberg.
Though SSI’s future plans are still unfolding, Sutskever and his team’s dedication to developing safe superintelligent AI makes the company one to watch closely in the coming years.
Sources
Sign Up for the Daily AI Pulse
One email a day. All the stories that matter.
By signing up, you agree to our Terms of Use and acknowledge the data practices in our Privacy Policy. You may unsubscribe at any time.
Daniel ReedAI Research, Safety & Ethics AnalystDaniel Reed currently works as an AI Research, Safety & Ethics Analyst at Aiholics, writing about how changes in artificial intelligence are affecting and will affect scholarship,…



