OpenAI Sets Up New Security Oversight Team


By Rashmi Ramesh (rashmiramesh_)

OpenAI recently set up a committee to make “critical” safety and security decisions for all of its projects, as the technology giant begins to train its next artificial intelligence model

CEO Sam Altman and board members Adam D’Angelo, Nicole Seligman and Bret Taylor will lead the committee, the company said. Over the next three months, the team will evaluate OpenAI’s processes and safeguards and recommend improvements. OpenAI will share an update on which recommendations it chooses to adopt “in a manner that is consistent with safety and security.”

The committee will also comprise technical and policy leads at the company, including head of preparedness; Aleksander Mądry,  head of safety systems; Lilian Weng, head of alignment science; John Schulman, head of security; ,  Matt Knight and chief scientist Jakub Pachocki,. The company plans to consult external security and technical experts including former White House cybersecurity official Rob Joyce.

The committee’s formation comes after OpenAI reportedly disbanded its “superalignment” security team dedicated to preventing AI systems from going rogue.

The superalignment safety team’s leaders – OpenAI co-founder Ilya Sutskever and Jan Leike – quit the company over their misalignment on the approach to security, as did policy researcher Gretchen Kreuger.

Both Sutskever and Leike worked on addressing the long-term safety risks facing the company and the technology, and Leike criticized OpenAI’s lack of support for the superalignment security team in a social media post. “Over the past years, safety culture and processes have taken a backseat to shiny products,” Leike said. Sutskever was among the board members who in November removed Altman from OpenAI only to see him reinstated as CEO five days later. Krueger said she decided to resign a few hours before her other two colleagues did, as she shared their security concerns.

Leave a Response

bahis canlı casino siteleri canlı bahis siteleri