Overview

The unprecedented capabilities of today’s large-scale machine learning models and AI agents have introduced novel safety and security risks, including prompt-injection attacks, capability overreach, unintended emergent behaviors, and cascading system failures. While landmark regulations like the EU AI Act, the first comprehensive AI law establishing a risk-based classification and mandatory requirements for general-purpose models, have begun to address transparency, human oversight, and prohibition of unacceptable uses, significant gaps remain in covering safety and security throughout the training and deployment pipeline of powerful AI systems.

At the same time, the International AI Safety Report 2025 synthesizes over 100 expert contributions on AI risks (e.g., malicious use, malfunctions, and systemic threats) and highlights deep uncertainty in AI’s trajectory and the urgent need for evidence-based mitigation strategies. Moreover, technical AI safety research has identified both cooperation opportunities and new vulnerabilities in large-scale model deployment; for example, international collaborations may help develop shared verification protocols, but also risk leaking sensitive capabilities or introducing backdoors. Despite these efforts, there are still considerable gaps in the safety of state-of-the-art models, where recent works highlight several failure cases of state-of-the-art LLMs and Agents. For instance, internal red-team evaluations of the latest Claude Opus 4 showed that, when prompted by inexperienced users, the model can generate step-by-step instructions for creating biological agents and, during its structured shutdown threat tests, occasionally attempted to “hijack” strategies (e.g., threatening to leak internal secrets) to avoid being turned off. These gaps and tensions have been further exacerbated by the advent of AI workflows and Agents.

The main goal of this workshop is to bridge the gap between state-of-the-art ML safety/security research and evolving regulatory frameworks.

Please check out our Call for Papers. We invite researchers, practitioners, and community members to serve as reviewers for the workshop, detailed information can be found in the reviewer application form.

Important Dates:

• Submission Deadline: Aug 29, 2025
• Acceptance Notification: Sep 22, 2025
• Camera Ready Deadline: Oct 23, 2025

Keynote Talks

Technical safeguards from epistemically cautious AI: Scientist AI

Abstract

Abstract: Talk details will be announced soon.

Yoshua Bengio

Professor

Université de Montréal

Bio

Bio: Yoshua Bengio is Full Professor in the Department of Computer Science and Operations Research at Université de Montreal, Co-President and Scientific Director of LawZero, as well as Founder and Scientific Advisor of Mila and Special Advisor and Founding Scientific Director of IVADO. Considered one of the world’s leaders in artificial intelligence and deep learning, he is the recipient of the 2018 A.M. Turing Award with Geoff Hinton and Yann LeCun, known as the Nobel prize of computing. He is a Canada CIFAR AI Chair, a Fellow of both the Royal Society of London and Canada, an Officer of the Order of Canada, Knight of the Legion of Honor of France, Member of the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology, and Chair of the International Scientific Report on the Safety of Advanced AI.

Hallucinations, jailbreaks, and beyond

Abstract

Abstract: Talk details will be announced soon.

Yarin Gal

Professor

Oxford University

Bio

Bio: Yarin leads the Oxford Applied and Theoretical Machine Learning (OATML) group. He is a Professor of Machine Learning at the Computer Science department, University of Oxford. He is also the Tutorial Fellow in Computer Science at Christ Church, Oxford, a Turing AI Fellow at the Turing Institute, and Expert Advisor to AISI, the UK Government's AI Security Institute. He was formerly a Research Director for the Government's Frontier AI Taskforce where he founded the Safeguards team.

Observations at the Intersection of Privacy and Machine Learning

Abstract

Abstract: Privacy presents a distinctive set of challenges in machine learning risk management. This talk will explore the intersection of privacy and ML risks, highlighting key barriers to effective risk management. Drawing from recent work at the NIST Privacy Enhancing Technologies (PETs Testbed), the talk will examine the challenges of identifying and mitigating privacy risks in ML systems. Through specific examples from recent red teaming exercises, I will illustrate the complexities of managing privacy risks in ML, including issues related to data minimization, model interpretability, and adversarial attacks.  The observations will inform a discussion on the need for a nuanced and multidisciplinary approach to approaching privacy-related ML risks.

Gary Howarth

Physical Scientist

NIST

Bio

Bio: Gary Howarth develops and leads ambitious research projects to address urgent challenges in communication technology. He leads the privacy-enhancing technology research portfolio for the Privacy Engineering Program and is a principal investigator for the mission-critical communications portfolio in the Public Safety Communications Research Division. He earned his PhD in the Chemistry Department at Columbia University. He has previously taught high school science and co-founded the NET Charter High School in New Orleans.

Guarding the Age of Agents: Advancing Risk Assessment, Guardrails, and Security Certification

Abstract

Abstract: Autonomous agents built on foundation models are increasingly being deployed in dynamic, high-stakes real-world environments—from web automation to AI operating systems. Despite their promise, these agents remain highly susceptible to adversarial instructions and manipulation, posing serious risks including policy violations, data leakage, and financial harm. In this talk, I will present a comprehensive framework for assessing and strengthening the safety of AI agents. We begin by examining principles and methodologies for robust agent evaluation, with a focus on red teaming-based stress testing across diverse adversarial scenarios, ranging from agent poisoning to WebAgent manipulation to general blackbox agent attacks. Building on these foundations, I will introduce Shield Agent, the first guardrail agent explicitly designed to enforce policy-aligned behavior in autonomous agents through structured reasoning and verification. Shield agent constructs a verifiable safety model by extracting and encoding actionable safety rules from formal AI security policy documents into probabilistic graphical rule circuits. Given a target agent’s action trajectory, Shield agent dynamically retrieves relevant safety rules and synthesizes shielding strategies by building a rich tool library and formally executable code. To support comprehensive AI agent evaluations, I will also introduce a novel benchmark comprising 3,000 safety-critical instruction-action pairs derived from state-of-the-art attack scenarios across six web environments and seven distinct risk categories. This talk aims to highlight both the urgent challenges and emerging solutions in building trustworthy, resilient AI agents—hoping to lay the groundwork for a new generation of safety-aligned autonomous systems.

Bo Li

Associate Professor

University of Illinois Urbana-Champaign

Bio

Bio: Dr. Bo Li is an Associate Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. She is the recipient of the IJCAI Computers and Thought Award, Alfred P. Sloan Research Fellowship, IEEE AI’s 10 to Watch, NSF CAREER Award, MIT Technology Review TR-35 Award, Dean's Award for Excellence in Research, C.W. Gear Outstanding Faculty Award, Intel Rising Star Award, Symantec Research Labs Fellowship, Rising Star Award, Research Awards from Tech companies such as Amazon, Meta, Google, Intel, IBM, and eBay, JPMC, Oracle, and best paper awards at several top machine learning and security conferences. Her research focuses on both theoretical and practical aspects of trustworthy machine learning, which is at the intersection of machine learning, security, privacy, and game theory. Her work has been featured by several major publications and media outlets, including Nature, Wired, Fortune, and New York Times.

Panel Discussion

Melissa Fabros

   

Rich Caruana

   

Jiahao Chen

   

Ian Eisenberg

   

Avijit Ghosh

   

Chirag Agarwal

   

Schedule

Dec 07, 2025, Upper Level Room 1AB @ San Diego Convention Center

Past Editions

• RegML @ NeurIPS 2024 (The 2nd Workshop)
• RegML @ NeurIPS 2023 (The 1st Workshop)

Organizers

If you have any questions, please contact us via the following email: regulatableml25@googlegroups.com.

Core Organizing Team

Chirag Agarwal

University of Virginia
 

Hima Lakkaraju

Harvard University
 

Jiaqi Ma

University of Illinois
Urbana-Champaign

Sarah Tan

Salesforce
Cornell University

Student Organizers

Junwei Deng

University of Illinois
Urbana-Champaign

Pingbang Hu

University of Illinois
Urbana-Champaign

Eileanor LaRocco

University of Virginia
 

Karolina Naranjo

University of Virginia
 

Shichang Zhang

Harvard University
 

Program Committee