Skip to main content

Advertisement

SpringerLink
Log in

Excavating AI: the politics of images in machine learning training sets

  • Original Article
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

A Correction to this article was published on 23 November 2021

This article has been updated

Abstract

By looking at the politics of classification within machine learning systems, this article demonstrates why the automated interpretation of images is an inherently social and political project. We begin by asking what work images do in computer vision systems, and what is meant by the claim that computers can “recognize” an image? Next, we look at the method for introducing images into computer systems and look at how taxonomies order the foundational concepts that will determine how a system interprets the world. Then we turn to the question of labeling: how humans tell computers which words will relate to a given image. What is at stake in the way AI systems use these labels to classify humans, including by race, gender, emotions, ability, sexuality, and personality? Finally, we turn to the purposes that computer vision is meant to serve in our society—the judgments, choices, and consequences of providing computers with these capacities. Methodologically, we call this an archeology of datasets: studying the material layers of training images and labels, cataloguing the principles and values by which taxonomies are constructed, and analyzing how these taxonomies create the parameters of intelligibility for an AI system. By doing this, we can critically engage with the underlying politics and values of a system, and analyze which normative patterns of life are assumed, supported, and reproduced.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

Not applicable.

Code availability

Not applicable.

Change history

  • 25 November 2021

    The Editor-in-Chief has removed Figure 1 and replaced Figure 2a due to copyright and consent concerns

  • 23 November 2021

    A Correction to this paper has been published: https://doi.org/10.1007/s00146-021-01301-1

Notes

  1. Minsky currently faces serious allegations related to convicted pedophile and rapist Jeffrey Epstein. Minsky was one of several scientists who met with Epstein and visited his island retreat where underage girls were forced to have sex with members of Epstein’s coterie. As scholar Meredith Broussard observed, there is a a broader culture of exclusion and hostility that became endemic in AI: “as wonderfully creative as Minsky and his cohort were, they also solidified the culture of tech as a billionaire boys’ club. Math, physics, and the other “hard” sciences have never been hospitable to women and people of color; tech followed this lead.” See Broussard (2018).

  2. See Crevier D (1993).

  3. Minsky gets the credit for this idea, but clearly Papert, Sussman, and teams of “summer workers” were all part of this early effort to get computers to describe objects in the world. See Papert SA (1966). As he wrote: “The summer vision project is an attempt to use our summer workers effectively in the construction of a significant part of a visual system. The particular task was chosen partly because it can be segmented into sub-problems which allow individuals to work independently and yet participate in the construction of a system complex enough to be a real landmark in the development of ‘pattern recognition’.

  4. Russell SJ (2010).

  5. In the late 1970s, Ryszard Michalski wrote an algorithm based on “symbolic variables” and logical rules. This language was very popular in the 1980s and 1990s, but, as the rules of decision-making and qualification became more complex, the language became less usable. At the same moment, the potential of using large training sets triggered a shift from this conceptual clustering to contemporary machine-learning approaches. See Michalski R (1980).

  6. There are hundreds of scholarly books in this category, but for a good place to start, see Mitchel WJT (2007).

  7. As described in the AI Now Report 2018, this classification of emotions into six categories has its root in the work of the psychologist Paul Ekman. “Studying faces, according to Ekman, produces an objective reading of authentic interior states—a direct window to the soul. Underlying his belief was the idea that emotions are fixed and universal, identical across individuals, and clearly visible in observable biological mechanisms regardless of cultural context. But Ekman’s work has been deeply criticized by psychologists, anthropologists, and other researchers who have found his theories do not hold up under sustained scrutiny. The psychologist Lisa Feldman Barrett and her colleagues have argued that an understanding of emotions in terms of these rigid categories and simplistic physiological causes is no longer tenable. Nonetheless, AI researchers have taken his work as fact, and used it as a basis for automating emotion detection.” Whitaker M et al. (2018). See also Barrett LF et al. (2019).

  8. See, for example, Leys R (2010). Leys has offered a number of critiques of Ekman’s research program, most recently in Ruth Leys, The Ascent of Affect: Genealogy and Critique Chicago and London: University of Chicago Press, 2017). See also Barret LF (2006); Siegel EH et al. (2018).

  9. Fei-Fei Li, as quoted in Gershgorn D (2017).

  10. Markoff J (2012).

  11. Their paper can be found here: Krizhevsky et al. (2012).

  12. Released in the mid-1980s, this lexical database for the English language can be seen as a thesaurus that defines and groups English words into synsets, i.e., sets of synonyms. https://wordnet.princeton.edu This project takes place in a broader history of computational linguistics and natural-language processing NLP), which developed during the same period. This subfield aims at programming computers to process and analyze large amounts of natural language data, using machine-learning algorithms.

  13. See Bowker GC (2000); Bechmann et al. (2019).

  14. These are some of the categories that have now been entirely deleted from ImageNet as of January 24, 2019.

  15. For an account of the politics of classification in the Library of Congress, see Berman S (1971).

  16. We’re drawing in part here on the work of Lakoff (2012).

  17. See Deng et al. (2009).

  18. Quoted in Sekula A (1986).

  19. Ibid; for a broader discussion of objectivity, scientific judgment, and a more nuanced take on photography’s role in it, see Daston et al. (2010).

  20. UTKFace (2019).

  21. See Edwards and Gabriellecht (2010). Earlier classifications used in the 1950 Population Act and Group Areas Act used four classes: “Europeans, Asiatics, persons of mixed race or coloureds, and ‘natives’ or pure-blooded individuals of the Bantu race” Bowker and Star, 197). Black South Africans were required to carry pass books and could not, for example, spend more than 72 h in a white area without permission from the government for a work contract 198).

  22. Bowker and Star, 208.

  23. See Davis FJ (2001).

  24. See Buolamwini and Gebru (2018).

  25. Merler et al. (2019).

  26. Webscope | Yahoo Labs (2019).

  27. Solon O (2019).

  28. Gould (1996). The approach of measuring intelligence based on skull size was prevalent across Europe and the US. For example, in France, Paul Broca and Gustave Le Bon developed the approach of measuring intelligence based on skull size. See Broca (1864). Bon (1881). See Justin (1943).

  29. Fiure Eight (2019).

  30. The authors made a backup of the ImageNet dataset prior to much of its deletion.

  31. Their “MegaPixels” project is here: https://megapixels.cc/

  32. Satisky (2019).

  33. 2nd Unconstrained Face Detection and Open Set Recognition Challenge (2015).

  34. Locker M (2019).

  35. Murgia M (2019).

  36. Locker, “Microsoft, Duke, and Stanford Quietly Delete Databases”.

  37. Full video here: Singh (2018).

  38. Melendez (2018).

  39. Vincent (2018).

  40. Ibid.

  41. Gould, The Mismeasure of Man, 140.

References

Download references

Acknowledgements

An earlier version of this article was originally published on September 19, 2019 at http://www.excavating.ai. Thanks to all those who have given editorial feedback, technical support, research contributions, and conversations on these issues over the years, including Arvind Narayanan, Daniel Neves, Varoon Mathur, Olga Russakovsky, Leif Ryge, Léa Saint-Raymond, and Kiran Samuel., Additional thanks to Mario Mainetti and Carlo Barbatti and all the staff at the Fondazione Prada, and to Alona Pardo and the staff at the Barbican Centre. The images in this essay and many more are part of the Training Humans exhibition, at the Fondazione Prada Osservatorio in Milan from September 12, 2019 through February 24, 2020.

Funding

None.

Author information

Authors and Affiliations

  1. University of Southern California, Annenberg School, Microsoft Research New York, New York, USA

    Kate Crawford

  2. The University of Georgia, Athens, Greece

    Trevor Paglen

Authors
  1. Kate Crawford
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Trevor Paglen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kate Crawford.

Ethics declarations

Conflicts of interest

The authors declare no conflict of interest other than the professional affiliations as listed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Crawford, K., Paglen, T. Excavating AI: the politics of images in machine learning training sets. AI & Soc 36, 1105–1116 (2021). https://doi.org/10.1007/s00146-021-01162-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00146-021-01162-8

Keywords

Search

Navigation