Overview

With the increasing deployment of machine learning in diverse applications affecting our daily lives, ethical and legal implications are rising to the forefront. Governments worldwide have responded by implementing regulatory policies to safeguard algorithmic decisions and data usage practices. However, there appears to be a considerable gap between current machine learning research and these regulatory policies. Translating these policies into algorithmic implementations is highly non-trivial, and there may be inherent tensions between different regulatory principles.

This workshop aims to provide a platform for: i) discussing various algorithmic, technical, and policy challenges that arise when operationalizing various guidelines outlined in existing regulatory frameworks, and ii) finding solutions to mitigate and address these challenges.

Please check out our Call for Papers.

Important Dates:

• Abstract Deadline: Aug 25 Sep 7, 2024
• Submission Deadline: Aug 30 Sep 12, 2024
• Acceptance Notification: Oct 14, 2024

Keynote Talks

Red Teaming to Identify & Mitigate Safety Risks in AI Systems

Abstract

Abstract: Dr. Rumman Chowdhury is a Responsible AI Fellow at the Berkman Klein Center for Internet & Society at Harvard University. She is also CEO and Co-Founder, Humane Intelligence. She is also a Research Affiliate at the Minderoo Center for Democracy and Technology at Cambridge University and a visiting researcher at the NYU Tandon School of Engineering. She is an active contributor to discourse around responsible technology with bylines in the Atlantic, Forbes, Harvard Business Review, Sloan Management Review, MIT Technology Review and VentureBeat. She was recently named Time Magazine's 100 Most Influential People in AI.

Rumman Chowdhury

CEO and co-Founder

Humane Intelligence

Bio

Bio: Dr. Rumman Chowdhury is a Responsible AI Fellow at the Berkman Klein Center for Internet & Society at Harvard University. She is also CEO and Co-Founder, Humane Intelligence. She is also a Research Affiliate at the Minderoo Center for Democracy and Technology at Cambridge University and a visiting researcher at the NYU Tandon School of Engineering. She is an active contributor to discourse around responsible technology with bylines in the Atlantic, Forbes, Harvard Business Review, Sloan Management Review, MIT Technology Review and VentureBeat. She was recently named Time Magazine's 100 Most Influential People in AI.

Why and how to regulate Frontier AI?

Abstract

Abstract: Opinions differ on the horizon but a majority of ML researchers believe that we are on track to achieve human-level AI across most cognitive tasks. What are the different risks associated with these future advances and how can we evaluate and mitigate them? The author is chairing the International Scientific Report on the Safety of Advanced AI which seeks to synthesize the scientific literature on this subject and inform policy-makers. In this context, regulation seems necessary to protect the public against these risks, to prevent economically expensive backlash against the technology in case of accidents, to level the playing field and not reward the least safety-conscious AI labs. The challenge is that AI methods are advancing rapidly and that the science of AI safety is still nascent and lacks answers about how to provide strong safety assurances. We will discuss why this calls for regulation that is future-proof and not prescribing particular evaluation and mitigation methodologies but incentivizing corporations to invest more in the required R&D..

Yoshua Bengio

Professor

Université de Montréal

Bio

Bio: Yoshua Bengio is Full Professor in the Department of Computer Science and Operations Research at U. Montreal, as well as the Founder and Scientific Director of Mila and the Scientific Director of IVADO. He also holds the Canada CIFAR AI Chair. He is one of the world's leading experts in artificial intelligence and deep learning. In 2018, he received the A.M. Turing Award, which is considered the "Nobel prize of computing." He is a Fellow of the Royal Society of London and Canada, an Officer of the Order of Canada, a Knight of the Legion of Honor of France, and a member of the UN's Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology. Concerned about the social impact of AI, he actively contributed to the Montreal Declaration for the Responsible Development of Artificial Intelligence and now devotes himself to reducing the catastrophic risks of future AI, currently chairing the International Scientific Report on the Safety of Advanced AI.

The Challenges of Pre-Deployment Regulability for General-purpose AI Systems

Abstract

Abstract: Policymakers and key stakeholders place significant emphasis on pre-deployment evaluations and model-based safeguards for general-purpose AI systems. However, these evaluations suffer from a wide range of challenges. In particular, they do not provide any guarantees about downstream model behaviors, especially under an adversarial model. This talk will explore these challenges, as well as paths forward.

Peter Henderson

Assistant Professor

Princeton University

Bio

Bio: Peter Henderson is an Assistant Professor at Princeton University with appointments in the Department of Computer Science and the School of Public and International Affairs. Previously, he received a JD and PhD in Computer Science at Stanford University. His research focuses on the intersection of AI and law. This includes building AI systems that work for the public good, making AI systems safe, and making sure that the law shapes positive outcomes for AI. His work has been covered by major media outlets like the New York Times and The Wall Street Journal. And has been cited by policymakers and courts.

Interactive Discussion on A Path for Science‑ and Evidence‑based AI Policy

Abstract

Abstract: As ML is becoming ubiquitous and creating new and impressive artifacts, regulatory agencies around the world are grappling with the unique properties of this new category of technology. In order to properly address these challenges, we argue for an approach centered on the dual pillars of rights and transparency to ensure that the technology is subject to the appropriate democratic governance. We outline some of the recent developments and proposals made by policymakers in this direction, how they connect to AI, and provide both organizational and technical tools to support well-informed regulation aligned with technology development now and in the future.

Dawn Song

Professor

University of California, Berkeley

Bio

Bio: Professor Song is the Director of Berkeley RDI; she works in AI, AI safety & security, and decentralization. Her work has won numerous awards, including MacArthur Fellowship,  Guggenheim Fellowship, more than 10 Test-of-Time Awards and Best Paper Awards at top Machine Learning and Computer Security conferences,  She is ranked the most cited scholar in computer security. She is also a serial entrepreneur, having founded multiple successful startups, named on the Female Founder 100 List by Inc. and Wired25 List of Innovators.

Rishi Bommasani

Society Lead

Stanford University

Bio

Bio: Rishi Bommasani is the Society Lead at the Stanford Center for Research on Foundation Models. He researches the societal impact of foundation models and advances evidence-based AI policy. Rishi's work has been featured in The Atlantic, MIT Technology Review, Nature, The New York Times, Quanta, Reuters, Science, The Wall Street Journal and The Washington Post. He serves as one of the chairs of EU AI Act Code of Practice and authors of the International Scientific Report on the Safety of Advanced AI.

Real World Matters: What Actually Happens When People Use AI? The NIST Assessing Risks and Impacts of AI (ARIA) Program

Abstract

Abstract: ARIA (Assessing Risks and Impacts of AI) is a NIST evaluation-driven research program to develop measurement methods that can account for AI’s risks and impacts in the real world. The program establishes an experimentation environment to gather evidence about AI’s risks to the public under controlled real-world conditions. In contrast to current approaches that rely on probabilities and predictions of what might happen, ARIA will enable direct observation of AI system behaviors and impacts on users. ARIA pairs people with AI applications in experimental scenarios designed around specific AI risks, and studies the results. Applications are submitted to NIST from around the globe and evaluated based on whether risks materialized in the resulting interactions, and the magnitude and degree of resulting impacts. Participating teams will learn whether their applications can maintain functionality across the varying contexts of the test environment.

Reva Schwartz

Research Scientist

National Institute of Standards and Technology

Bio

Bio: Reva Schwartz is a research scientist at the National Institute of Standards and Technology’s (NIST) Information Technology Laboratory (ITL) where she serves as Principal Investigator on Bias in Artificial Intelligence and leads ARIA, a research program which advances the measurement of AI’s risks to people and society. She previously served as forensic scientist for almost 15 years at the United States Secret Service and adjunct researcher at the Johns Hopkins University Human Language Technology Center of Excellence. \n Reva’s background is in linguistics and human language technology. A socio-technical researcher, she focuses on real world testing methodologies and people-centered approaches for AI measurement. She has advised federal agencies about how experts interact with automation to make sense of information in high-stakes settings.

Panel Discussion

Gillian K. Hadfield

Johns Hopkins
University
 

Wan Sie Lee

Singapore's IMDA
   

Surdas Mohit

ISED Canada
Canada AI
Safety Institute

Sella Nevo

RAND's Meselson
Center on AI
Security and Biosecurity

Rob Reich

Stanford University
US AI Safety Institute
 

Savannah Thais

Columbia University
   

Schedule

Dec 15, 2024; Vancouver Convention Centre (EAST MR 13)

Organizers

If you have any questions, please contact us via the following email: regml-2024@googlegroups.com.

Core Organizing Team

Jiaqi Ma

University of Illinois
Urbana-Champaign

Chirag Agarwal

University of Virginia
 

Sarah Tan

Salesforce
Cornell University

Hima Lakkaraju

Harvard University
 

Student Organizers

Usha Bhalla

Harvard University
 

Zana Bucinca

Harvard University
 

Junwei Deng

University of Illinois
Urbana-Champaign

Alex Oesterling

Harvard University
 

Shichang Zhang

Harvard University
 

Reviewers