Is AI Sexist? Wellesley Computer Science Professor Looks Into a Recent Amazon Recruiting Experiment
This article originally appeared on The Spoke, the blog of the Madeleine Korbel Albright Institute for Global Affairs, and was written by Eni Mustafaraj, assistant professor of computer science at Wellesley.
Amazon recently scrapped an experimental artificial intelligence (AI) recruiting tool that was found to be biased against women. At this point, I hope you might have a few questions, such as: What is an AI recruiting tool and how does it work? Why was it biased against women? I’ll try to answer them for you in the following.
The AI Recruiting Tool
You have certainly heard of human recruiters. They are matchmakers between employers and potential employees. They travel, send cold emails, and “network” at conferences and job fairs. When recruiters make a successful match, they get paid, sometimes by one party, sometimes by both. As you can see, this matchmaking dance is often expensive and time-consuming. Surely technology can help, right? A human recruiter can review at most a few dozen applicants per day, before she gets tired. In contrast, artificial intelligence can “read” thousands of applications in seconds and rank them based on desired criteria, showing the most promising candidates at the top. Understandably then, compared to a human recruiter, an AI recruiter would be more time- and cost-efficient. And now that the human recruiter doesn’t need to sift through and rank candidates, she can devote her time to reaching out to the best candidates and wooing them to accept an offer. What nice teamwork between the human and AI recruiters!
Unfortunately, things are never so simple. How can we ensure that the AI recruiter is being fair to all candidates? Can it offer explanations for why it didn’t suggest any women for a certain job opening? To answer these new questions, we need to understand how the AI tool “learns” to do its job.
It all starts with a big “training” set of job applications. For years, companies have been requiring job applicants to submit all their materials online. For example, if you have been on the academic job market, you were probably asked to upload your resume, cover letter, and letters of recommendation in a website like AcademicJobsOnline.org. Big corporations like Amazon and Google, unlike universities, run their own job application sites. Therefore, over time, they have amassed thousands and thousands of application materials, all in electronic form. Additionally, they have recorded which applicants were successful in their job hunts. Thus, they have examples of the materials submitted by applicants who were hired and by applicants who were rejected. This information is then given to the AI tool to “learn” the characteristics that reflect a successful candidate. In the case of Amazon’s tool, the AI “learned” that words like “executed” and “captured” in a resume correlate with success. Meanwhile, it also “learned” that the presence of a phrase like “women’s” (as in “women’s chess captain”) correlates with rejection, and so the corresponding resume was downgraded.
The full story can be found on The Spoke website.
