Through a Glass, Darkly

Artificial Intelligence and the Problem of Opacity

Bibliographic Data

ID9529373
AuthorsSimon Chesterman (0000-0002-3599-4573, National University of Singapore Faculty of Law, Singapore, corresponding author)
Year2021
Volume69
Issue2
Pages271-294
Publication date2021-11-08
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueThe American Journal of Comparative Law (JOURNAL)
Journal identifiersISSN: 0002-919X • E-ISSN: 2326-9197
PublisherOxford University Press (PUBLISHER • GB)
DOI10.1093/ajcl/avab012
OpenAlexW3157512024
LanguageEN
Citations received10

As computer programs become more complex, the ability of non-specialists to understand how a given output has been reached diminishes. Opaqueness may also be built into programs to protect proprietary interests. Both types of systems are capable of being explained, either through recourse to experts or an order to produce information. Another class of system may be naturally opaque, however, using deep learning methods that are impossible to explain in a manner that humans can comprehend. An emerging literature describes these phenomena or specific problems to which they give rise, notably the potential for bias against specific groups. Drawing on examples from the United States, the European Union, and China, this Article develops a novel typology of three discrete regulatory challenges posed by opacity. First, it may encourage—or fail to discourage—inferior decisions by removing the potential for oversight and accountability. Second, it may allow impermissible decisions, notably those that explicitly or implicitly rely on protected categories such as gender or race in making a determination. Third, it may render illegitimate decisions in which the process by which an answer is reached is as important as the answer itself. The means of addressing some or all of these concerns is routinely said to be through transparency. Yet, while proprietary opacity can be dealt with by court order and complex opacity through recourse to experts, naturally opaque systems may require novel forms of “explanation” or an acceptance that some machine-made decisions cannot be explained—or, in the alternative, that some decisions should not be made by machine at all

Accountability · Business · Class (philosophy) · European union · Law and economics · Opacity · Order (exchange) · Political science · Process (computing) · Sociology · Transparency (behavior) · Typology · Artificial Intelligence · Computer Science · Criminal Law and Evidence · Education, Law, and Society · Law · Law, AI, and Intellectual Property

Tools

Open DOI