OpenAI to fight AI’s “catastrophic risks”

At the end of October 2023, OpenAI announced the creation of a team of experts, named “Preparedness”, dedicated to fighting the “catastrophic” risks associated with AI. “Frontier AI models, which will exceed the capabilities currently present in the most advanced existing models, have the potential to benefit all of humanity. But they also pose increasingly severe risks,” said the ChatGPT publisher.

The team, led by Aleksander Madry, an AI expert from MIT, will focus on three main questions:

• “When purposefully misused, just how dangerous are the frontier AI systems we have today and those coming in the future?”
• “If frontier AI model weights were stolen, what exactly could a malicious actor do?”
• “How can a framework that monitors, evaluates, predicts, and protects against the dangerous capabilities of frontier AI systems be built?”

Specifically, Preparedness will study several risks associated with AI models:

• individualized persuasion (or matching the content of a message to what the recipient wants to hear);
• overall cybersecurity;
• autonomous replication and adaptation (AI changing itself on its own)
• extinction-level threats like chemical, biological, radiological, and nuclear attacks

OpenAI also launched the “AI Preparedness Challenge”. The company is offering up to $25,000 to the top 10 submissions of probable, but potentially catastrophic misuses of OpenAI.