Abstract

The machine learning community progresses by (respectively despite) improving the “accuracy” of its systems. On the other hand, the EU AI Act explicitly refers to “accuracy” as parts of its compliance measures for high-risk AI systems. Are we talking about related fictions of accuracy in both perspectives? Respectively, in which ways do those fictions relate? This work presents “accuracy” as a case-study for differing requirements of social worlds, the technological machine learning community and the community of lawmakers. While competition on accuracy is acknowledged a major driving force of technological development, machine learning scholars simultaneously recognize accuracy’s shortcomings when it comes to substantial claims on the usefulness and effectiveness of machine learning systems. The legal counterpart embraces vagueness around the term of accuracy, leaving interpretative flexibility for technological and societal changes. At the same time, accuracy is a core element of compliance with respect to the EU AI Act. We elaborate on three main dimensions of tension, (a) adequacy to purpose, (b) individualization and (c) objective versus certificate, to show that the two communities project disparate, and sometimes even contradicting, expectations on accuracy. We conclude that both of the communities, legal or technical, lack a precise understanding of ``accuracy’’ beyond the contextual boundaries of the community. We further highlight the danger of enacting “accuracy” as if it were a single notion. Instead, our multiperspective approach leads to standardization requests and technical challenges to be solved by the respective communities.

About the Speaker

Rabanus is a PhD Student in the Foundations of Machine Learning Systems group at the University of Tübingen where he works on the implication of probability theoretical assumptions in machine learning and the quality of forecasts. As part of this endeavour he is interested in exploring the interplay of how data is modelled and the way predictions on this data are evaluated. He is also a scholar in the International Max Planck Research School for Intelligent Systems at Tübingen and was a research scholar at University of Pennsylvania with Prof Aaron Roth.