Trustworthy ML Regulation as a Principal-Agent Problem
Mohammad Yaghinia, Patty Liub, Andrew Magnusona, Natalie Dullerudc, Nicolas Papernota
- aUniversity of Toronto and Vector Institute, Toronto, Canada
- bPrinceton University, Princeton, USA
- cStanford University, Palo Alto, USA
Papers open inside SciRah from the original publisher or repository; SciRah never hosts them.
Abstract
As ML-enabled systems become increasingly prevalent, so do their societal risks, such as excessive privacy and fairness violations.Mitigating such risks is frequently at odds with the system's overall performance.As a result, companies producing these systems and the public place different relative importance on these objectives-a case of misaligned incentives.Public regulators seeking to curb such risks are faced with designing regulations with a penalty structure that balances unmitigated societal risks and unnecessary financial burdens on a burgeoning industry.However, regulators often do not have access to the data or the training procedure of the model used by companies.Such access asymmetries can cause uncertainties in risk estimations that we show can lead to unintended over-regulation of law-abiding companies (1-7% drop in accuracy on three vision datasets).The access asymmetries that enabled unintended over-regulation are an artifact of the separation of the company and the regulator.This realization leads us to formulate ML Regulation as a Principal-Agent Problem (PAP).We provide a taxonomy of PAPs in this domain and tackle the problem of optimal penalty design in a reduced setting, which allows us to align the company's incentives with the public expectation of safety.
TL;DRA taxonomy of PAPs in this domain is provided and the problem of optimal penalty design in a reduced setting is tackled, which allows the company’s incentives with the public expectation of safety to align. (Semantic Scholar)