This Researcher Says AI Is Neither Artificial nor Intelligent
Kate Crawford, who holds positions at USC and Microsoft, says in a new book that even experts working on the technology misunderstand AI.
TECHNOLOGY COMPANIES LIKE to portray artificial intelligence as a precise and powerful tool for good. Kate Crawford says that mythology is flawed. In her book Atlas of AI, she visits a lithium mine, an Amazon warehouse, and a 19th-century phrenological skull archive to illustrate the natural resources, human sweat, and bad science underpinning some versions of the technology. Crawford, a professor at the University of Southern California and researcher at Microsoft, says many applications and side effects of AI are in urgent need of regulation.
Crawford recently discussed these issues with WIRED senior writer Tom Simonite. An edited transcript follows.
WIRED: Few people understand all the technical details of artificial intelligence. You argue that some experts working on the technology misunderstand AI more deeply.
KATE CRAWFORD: It is presented as this ethereal and objective way of making decisions, something that we can plug into everything from teaching kids to deciding who gets bail. But the name is deceptive: AI is neither artificial nor intelligent.
AI is made from vast amounts of natural resources, fuel, and human labor. And it’s not intelligent in any kind of human intelligence way. It’s not able to discern things without extensive human training, and it has a completely different statistical logic for how meaning is made. Since the very beginning of AI back in 1956, we’ve made this terrible error, a sort of original sin of the field, to believe that minds are like computers and vice versa. We assume these things are an analog to human intelligence, and nothing could be further from the truth.
You take on that myth by showing how AI is constructed. Like many industrial processes it turns out to be messy. Some machine learning systems are built with hastily collected data, which can cause problems like face recognition services more error prone on minorities.
We need to look at the nose to tail production of artificial intelligence. The seeds of the data problem were planted in the 1980s, when it became common to use data sets without close knowledge of what was inside, or concern for privacy. It was just “raw” material, reused across thousands of projects.
This evolved into an ideology of mass data extraction, but data isn’t an inert substance—it always brings a context and a politics. Sentences from Reddit will be different from those in kids’ books. Images from mugshot databases have different histories than those from the Oscars, but they are all used alike. This causes a host of problems downstream. In 2021, there’s still no industry-wide standard to note what kinds of data are held in training sets, how it was acquired, or potential ethical issues.
You trace the roots of emotion recognition software to dubious science funded by the Department of Defense in the 1960s. Arecent reviewof more than 1,000 research papers found no evidence a person’s emotions can be reliably inferred from their face.
Emotion detection represents the fantasy that technology will finally answer questions that we have about human nature that are not technical questions at all. This idea that’s so contested in the field of psychology made the jump into machine learning because it is a simple theory that fits the tools. Recording people’s faces and correlating that to simple, predefined, emotional states works with machine learning—if you drop culture and context and that you might change the way you look and feel hundreds of times a day.
That also becomes a feedback loop: Because we have emotion detection tools, people say we want to apply it in schools and courtrooms and to catch potential shoplifters. Recently companies are using the pandemic as a pretext to use emotion recognition on kids in schools. This takes us back to the phrenological past, this belief that you detect character and personality from the face and the skull shape.
You contributed to recent growth in research into how AI can have undesirable effects. But that field is entangled with people and funding from the tech industry, which seeks to profit from AI. Google recently forced out two respected researchers on AI ethics,Timnit GebruandMargaret Mitchell. Does industry involvement limit research questioning AI?
I can’t speak to what happened inside Google, but what I’ve seen is incredibly troubling. It’s important that we have researchers inside technology companies seeing how these systems work and publishing about it.
We’ve seen research focused too narrowly on technical fixes and narrow mathematical approaches to bias, rather than a wider-lensed view of how these systems integrate with complex and high stakes social institutions like criminal justice, education, and health care. I would love to see research focus less on questions of ethics and more on questions of power. These systems are being used by powerful interests who already represent the most privileged in the world.
Is AI still useful?
Let’s be clear: Statistical prediction is incredibly useful; so is an Excel spreadsheet. But it comes with its own logic, its own politics, its own ideologies that people are rarely made aware of.
And we’re relying on systems that don’t have the sort of safety rails you would expect for something so influential in everyday life. We have a regulation emergency: There are tools actually causing harm that are completely unregulated.
Do you see that changing soon?
We’re getting closer. We have Alondra Nelson in the White House Office of Science and Technology Policy, who has written about the fact that you cannot escape the politics of technology. And we’re starting to see a new coalition of activists and researchers seeing that the interrelatedness of capitalism and computation is core to climate justice, and labor rights, and racial justice. I’m optimistic.
This article first appeared in www.wired.com
