Ilya Sutskever, a co-founder of OpenAI, has launched Safe Superintelligence Inc. (SSI), a company dedicated to addressing a critical issue in technology: the creation and control of superintelligent AI. SSI’s mission, as outlined on its website, is to solve “the most important technical problem of our time” – ensuring that superintelligent AI systems are safe.
Sutskever is joined by OpenAI engineer Daniel Levy and former Y Combinator partner Daniel Gross. Together, they aim to make safety a priority in AI development, alongside capability. Sutskever has long been concerned about the potential benefits and risks of superintelligent AI. In a 2023 OpenAI blog post co-authored with Jan Leike, he discussed the challenges of controlling AI systems that surpass human intelligence. The post highlighted the limitations of current alignment techniques, such as reinforcement learning from human feedback, which depend on direct human supervision.
“Humans won’t be able to reliably supervise AI systems much smarter than us, and so our current alignment techniques will not scale to superintelligence,” the blog stated. “We need new scientific and technical breakthroughs.”
Sutskever’s departure from OpenAI to establish SSI signals his desire to focus solely on this issue. SSI’s website emphasizes its singular focus on safe superintelligent AI, free from the distractions of management overhead or product cycles. The company aims to attract top engineers and researchers to concentrate exclusively on this mission.
While details on SSI’s specific strategies are sparse, Sutskever has shared some insights in an interview with Bloomberg. He mentioned that SSI plans to integrate safety protocols within AI systems during development, rather than adding safeguards afterward.
“By safe, we mean safe like nuclear safety as opposed to safe as in ‘trust and safety,’” Sutskever explained to Bloomberg.
Though SSI’s future plans are still unfolding, Sutskever and his team’s dedication to developing safe superintelligent AI makes the company one to watch closely in the coming years.
