Data Science Checklists (Part 1): The Complexity of Putting Principles Into Practice

Josh Feldman · February 2, 2020

A recent trend in discussions surrounding the governance of machine learning (ML) algorithms is the drafting of so-called "AI principles". Organizations ranging from technology firms to government to civil society to multi-stakeholder initiatives have published documents outlining the values they believe should govern these technologies. In the context of Internet regulation, this phenomenon has been called digital constitutionalism in the sense that these documents might be the building blocks for future internet regulation, but lack any codified rules or enforcement mechanisms in their current formulation. These AI principles make foundational normative claims, but cannot demand compliance –AI principles are, for now, only politically charged pdfs. Thus, another question is implied by the very existence of these documents. How can a given set of principles best be put into practice?

Note: The following three blog posts will accompany a package called "checklist" that I developed. After installing the package, it will trigger a checklist in the command line whenever users call "git push" if they also insert a short shell script in their shell configuration (available here).

In part 1, we will introduce the trichotomy of simple, complicated, and complex problems and establish that implementing a set of AI principles should be thought of as a complex problem. Simple, complicated, and complex problems require different types of solutions and identifying that putting AI principles into practice is a complex problem will guide the rest of our discussion. In part 2, we will discuss whether any tools can help data scientists tackle this complex problem. Specifically, we look to the checklist, exemplified by the Surgical Safety Checklist described by Atul Gawande in the The Checklist Manifesto. The checklist designed by Gawande and his team achieved positive results in hospitals around the world, reducing complications and deaths in all eight hospitals they tested. It is rare for such a simple tool to be associated with measurable improvements on a difficult task in such a wide array of hospitals. Can these successes be transferred to the data science context? I will delay introducing the actual principles we seek to implement until part 3, when we will discuss the Design Justice Network Principles and translate these principles into a prototype checklist and an accompanying command line tool allowing data scientists to complete the checklist.

Simple, Complicated, and Complex Problems

As Gawande suggests, we can look to the work of Brenda Zimmerman and Sholom Glouberman for a framework of classifying problems presented in their report on the complexity of social policy. Zimmerman and Glouberman propose three problem types: simple, complicated, and complex. Note that these problem types are not necessarily mutually exclusive. A single problem may consist of simple, complicated, and complex components. The exemplary simple problem is cooking with a recipe. The aspect of this problem that makes it simple is the recipe. Almost anyone can learn to follow the recipe, and once learned, the output is predictable with success essentially guaranteed. Simple problems are ones for which accessible recipes exist. The example Zimmerman and Glouberman provide to describe a complicated problem is sending a rocket to the moon. Complicated problems consist of a collection of simple problems, but the whole is not reducible to its parts. The enormous number of recipes required to send a rocket to the moon requires experts, who have mastered a number of simple problems, and coordination between these experts. Nevertheless, even though solutions to complicated problems do not of appear as recipes in practice, they theoretically can be understood as an unimaginable number of recipes being executed in a coordinated fashion. Because of this underlying order, solutions to complicated problems can generally be reused with a high degree of certainty. After a team successfully lands a rocket on the moon, future astronauts on the team can be significantly more confident about the success of their own mission.

Complex problems, however, do not have this underlying recipe structure. Zimmerman and Glouberman illustrate complex problems with the challenge of raising a child. Raising a child and other complex problems are highly contingent, with each instance of the problem being significantly different from the next. Even if one raised a child with some degree of success, this does not guarantee that things will go well with the next one. Each child is different and, accordingly, requires different types of parenting. Zimmerman and Glouberman list some features of complex problems that can generate this contingency:

  • Dependence on local conditions;
  • Interdependence between different aspects of the problem;
  • Non-linear relationships and chaotic behaviour;
  • Ambiguity and uncertainty;
  • Changing conditions.

This list is not exhaustive, but rather seeks to communicate some elements of what complex problems are like.

The Complexity of Putting Principles into Practice

Putting AI principles into practice is a complex problem. The complexity can be illustrated by discussions surrounding the principle of requiring machine learning algorithms to be "fair". Creating a fair algorithm is complex because there is no universal criterion of fairness that can be applied to judge whether one has succeeded. This argument is made by Rodrigo Ochigame in "The Long History of Algorithmic Fairness ". He provides two arguments supporting the claim that fairness cannot be formalized, one historical and one theoretical. By tracing the history of algorithmic fairness from Aristotle to today, Ochigame historicizes the term. We observe algorithmic fairness take on different meanings depending on political circumstance and who holds power over whom. Most notable is the concept of actuarial fairness, that is, "that each person should pay for her own risk". The notion of actuarial fairness was promoted by insurance companies in the 1970s to suppress criticisms by American civil rights and feminist activists that risk classification was harming already marginalized communities. The insurance companies won both the political and epistemological fight, avoiding regulation and achieving social legitimacy for their definition of fairness. This case in not unique. The remainder of the historical overview retold by Ochigame demonstrates that notions of mathematical or algorithmic fairness are always grounded in economic or political motivations.

The histories of legal, political, and social theory support the conclusion that a mathematical formalism of fairness is impossible. Over and above the philosophies within these disciplines being motivated by contextual economic and political factors, these bodies of thought also contain contradictions. A mathematical formalism of fairness must choose between these ideologies. This choice ultimately results in the algorithm perpetuating the moral views held within the philosophical system that created the fairness definition. Any attempt to put the principle that algorithms should be fair into practice must negotiate the challenge of defining what is meant by a fair outcome in every decision made by the algorithm.

Hence, putting the principle of fairness into practice is neither simple nor complicated, but complex. It is not simple because there is no single recipe to achieve a fair outcome. Such a recipe does not exist because the very concept of fairness is historically contingent and requires choosing between a set of contradictory definitions. It is not complicated because even if one were to bring together the best social scientists, computer scientists, legal theorists, and philosophers from a variety of fields and coordinated their work perfectly, a replicable method of making fair algorithms would not emerge. In fact, such an effort would not even result in a universal definition of fairness, since as we have shown, experts hold varying and contradictory notions of what is fair. Putting fairness into practice is complex because each application requires reassessing the context and redefining what justice looks like. It is important to note that this contingency is not relativism or nihilism. Returning to the example of raising a child, even though success looks different for each child, this does not mean we give up on raising children well. We can still strive to create algorithms that reflect the principle of fairness, even if these principles must be reinterpreted and adapted to the case in question. This discussion has used fairness as a particularly contentious example of translating principles into practice, but in general we must always negotiate between the generalities of our principles and the thorny particularities of the case in question. Thus, the task of putting a set of AI principles into practice is a complex one. We continue this discussion in part 2, where we examine whether checklists can help data scientists handle the complexity of having their work align with a set of AI principles.

References:

A. Gawande, The Checklist Manifesto, New York: Metropolitan Books, 2010.
S. Glouberman and B. Zimmerman, "Complicated and Complex Systems: What Would Successful Reform of Medicare Look Like?," Commission on the Future of Health Care in Canada, Canada, 2002.
Access Now, "The Toronto Declaration: Protecting the rights to equality and non-discrimination in machine learning systems," Access Now, Toronto, 2018.
Partnership on AI, "Tenets," [Online]. Available: https://www.partnershiponai.org/tenets/.
Smart Dubai, "Artificial Intelligence Principles & Ethics," [Online]. Available: https://www.smartdubai.ae/initiatives/ai-principles-ethics.
L. Gill, D. Redeker and U. Gasser, "Towards Digital Constitutionalism? Mapping Attempts to Craft an Internet Bill of Rights," Berkman Center Research Publication No. 2015-15 , 2015.
A. B. Haynes, T. G. Weiser, W. R. Berry, S. R. Lipsitz, A.-H. S. Breizat, E. P. Dellinger, T. Herbosa, S. Joseph, P. L. Kibatala, M. C. M. Lapitan, A. F. Merry, K. Moorthy and R. Reznick, "A Surgical Safety Checklist to Reduce Morbidity and Mortality in a Global Population," The New England Journal of Medicine, vol. 360, no. 5, pp. 491-499, 2009.
R. Ochigame, The Long History of Algorithmic Fairness, 2019.
Google, "Artificial Intelligence at Google: Our Principles," [Online]. Available: https://ai.google/principles/.

Twitter, Facebook