EU AI Office Sets New AGI Safety Rules for Frontier AI Developers

The European AI Office releases new AGI safety guidelines requiring developers like OpenAI to submit safety cases for models exceeding specific compute limits.

BRUSSELS — The European AI Office on Wednesday released the "Guidelines on Systemic Risk Mitigation for Frontier AGI Models," a 240-page regulatory framework establishing the first mandatory safety standards for general-purpose artificial intelligence (GPAI). The document outlines the operational requirements for developers whose models exceed the compute threshold of 10²⁶ floating-point operations (FLOPs). This benchmark currently applies to high-capacity systems including OpenAI’s GPT-5, Anthropic’s Claude 4, and Mistral’s "Project Jupiter."

The guidelines mandate that developers submit a "Safety Case" to the AI Office 90 days before any public or commercial deployment. This filing must provide empirical evidence that residual risks of catastrophic misuse remain below a defined societal threshold. The AI Office defines catastrophic misuse as the facilitation of large-scale cyberattacks, the design of biological weapons, or the autonomous disruption of critical infrastructure.

Under the new rules, companies must report unforeseen emergent behaviors or security breaches involving model weights within 48 hours. Failure to comply with these transparency mandates carries financial penalties of up to €35 million or 7% of total worldwide annual turnover, whichever is higher. The AI Office also reserved the right to designate models below the 10²⁶ FLOPs threshold as "systemic" if they demonstrate significant qualitative impact or reach a viral scale of adoption within the European Single Market.

The release follows the full application of the EU AI Act, which entered into force in August 2024. European regulators spent 2025 conducting internal stress tests on both open-source and proprietary models to determine where voluntary commitments failed to address systemic threats. A February 2025 audit revealed that existing self-regulation did not adequately prevent models from generating polymorphic malware or bypassing standard safety filters.

The definition of General Purpose AI shifted throughout 2025 as "Agentic AI" gained prominence. These systems, capable of executing multi-step tasks without human intervention, forced the AI Office to expand its oversight beyond static chatbots. The rise of models capable of autonomous software modification and financial execution led to the inclusion of "kill-switch" protocols in the final guidelines issued today.

In November 2025, a leaked draft of these guidelines caused a 4% decline in major technology stocks as investors reacted to the proposed transparency requirements. The final version maintains the strict 90-day pre-deployment review period despite intense lobbying from Silicon Valley and the Frontier Model Forum. The AI Office now operates with a staff of 450 specialists, including 150 experts dedicated to adversarial red-teaming.

The guidelines introduce a Scientific Panel of 25 independent scientists, chaired by Dr. Yoshua Bengio. This panel will verify the safety claims made in developer "Safety Cases" and provide technical assessments to the AI Board, which consists of representatives from all 27 EU Member States. The panel has the authority to demand access to a model's training data and fine-tuning methodologies to ensure compliance with the new standards.

Energy transparency forms a central pillar of the new reporting requirements. Developers must disclose the total energy consumption of the training phase, measured in kilowatt-hours (kWh). They are also required to report the carbon intensity of each 1 million inferences, providing a standardized metric for the environmental cost of large-scale AI operations.

The "Red-Teaming Standards" section was revised following a shadow audit of a California-based lab last month. The audit discovered that internal testing had missed a vulnerability allowing the model to assist in creating self-replicating malicious code. Consequently, the guidelines now mandate third-party verification of all red-teaming results. Developers must prove that their models cannot be "jailbroken" to provide actionable instructions for illegal activities.

For models exhibiting "Agentic Autonomy," the guidelines require a hardware-level override. Any AI system capable of autonomous financial transactions or modification of its own source code must include a protocol that allows regulators or owners to terminate all active processes instantly. This measure aims to prevent runaway autonomous behavior in financial markets or digital infrastructure.

For full coverage, visit https://www.linos.ai/technology/eu-ai-office-agi-safety/

About Linos NEWS: Linos NEWS (https://www.linos.ai) delivers breaking news and in-depth analysis across politics, technology, business, science, health, world affairs, sports, and entertainment.

Media Contact

Linos NEWS

Linos NEWS

https://www.linos.ai

Keywords: technology
Share this press release:

Have your own news to share? Submit Press Release Free