The stated goal of many organizations in the field of artificial intelligence (AI) is to develop artificial general intelligence (AGI), an imagined system with more intelligence than anything we have ever seen. Without seriously questioning whether such a system can and should be built, researchers are working to create “safe AGI” that is “beneficial for all of humanity.” We argue that, unlike systems with specific applications which can be evaluated following standard engineering principles, undefined systems like “AGI” cannot be appropriately tested for safety. Why, then, is building AGI often framed as an unquestioned goal in the field of AI? In this paper, we argue that the normative framework that motivates much of this goal is rooted in the Anglo-American eugenics tradition of the twentieth century. As a result, many of the very same discriminatory attitudes that animated eugenicists in the past (e.g., racism, xenophobia, classism, ableism, and sexism) remain widespread within the movement to build AGI, resulting in systems that harm marginalized groups and centralize power, while using the language of “safety” and “benefiting humanity” to evade accountability. We conclude by urging researchers to work on defined tasks for which we can develop safety protocols, rather than attempting to build a presumably all-knowing system such as AGI.
Eugenics
By blaming poor children’s school learning failure on their brains, the billionaires are continuing a long pseudoscientific charade extending back to 19th century “craniology,” which used head shape-and-size to explain the intellectual inferiority of “lesser” groups, such as southern Europeans and blacks. When craniology finally was debunked in the early 20thcentury, psychologists devised the IQ test, which sustained the mental classification business. Purportedly a more scientific instrument, it was heavily used not only to continue craniology’s identification of intellectually inferior ethnic and racial groups, but also to “explain” the educational underachievement of black and poor-white students.
After decades of use, IQ tests were substantially debunked from the 1960s onward, but new, more neurologically complex, so-called brain-based explanations emerged for differing educational outcomes. These explanations conceived of the overall brain as normal, but contended that brain glitches impeded school learning and success. Thus entered “learning disabilities,” “dyslexia,”and “attention deficit hyperactivity disorder (ADHD)” as major neuropsychological concepts to (1) explain school failure, particularly for poor children, although the labels also extended to many middle-class students; and (2) serve as “scientific” justification for scripted, narrow, pedagogy in which teachers seemingly reigned in the classroom, but in fact, were themselves controlled by the prefabricated curricula.