Jul 29, 2022

9 Min Read REGISTER NOW Patrick Sturdivant, a software systems engineer who worked at USAA for 34 years and is blind, is generally a fan of artificial intelligence software. “I love AI when it's used properly and thoughtfully to include everyone,” he said. “AI can be a great equalizer for the disabled, because the computer can do so much that I can't do.” But he also worries about the dangers of AI inadvertently discriminating against people with disabilities during recruitment and hiring. “I do get scared that too much AI is used to filter through resumes and a lot of good people are excluded,” said Sturdivant, who is now principal strategy consultant at Deque Systems. “What scares me even more is the use of AI to screen video interviews to see who's going to make the next cut. There is too much potential there for developers to forget to include people of all backgrounds, all abilities, all races, all colors, all genders. And if they don't do it right, and if they don't test it correctly and perfect it, they're going to hurt some people.”  Most large employers, including banks, use AI throughout the hiring process to filter through thousands of applications to get to the smaller number of truly qualified candidates. This is sometimes called “predictive hiring.” Across the industry, 51% of banks said they use AI-powered analytics in decision making of all kinds, according to a survey conducted by Arizent that was published earlier this week . “More and more organizations are using different algorithms to try and create better matches of prospective employees to jobs in their organization,” said Haig Nalbantian, senior partner at Mercer and co-lead of Mercer's Workforce Sciences Institute. “There's been a big uptick.” This use of AI potentially could reduce or eliminate nepotism and bias from hiring decisions. But it could also perpetuate existing biases or introduce new forms of unfairness, including in the use of gamification, automated analysis of resumes, automated analysis of video interviews and AI-generated pre-employment tests. Gamification One AI-based technology Nalbantian has been seeing banks and other large companies use more in hiring is gamification software from companies like Pymetrics and Knack. These use games to measure potential employees’ “soft skills” and match them with job openings. In some cases, the games simulate actual work experience. “It might involve some problem solving, and the employer through this mechanism will be able to see, how does this individual tackle a problem? What does he or she look at first? Are they attentive to things happening on the screen that may not be central to what they're focused on? In what order do they address particular issues that arise? How do they react to surprises?” Nalbantian said. “They're trying to simulate the work experience to gauge how people actually function when they're forced to make choices in an environment that's gamified.” The providers of these programs argue that their technology can ensure a company is blind to race and gender. “In that way, it can offset some of the overt bias that may show up in traditional hiring methods, where I look at a resume and the name, the addresses and the background are all there,” Nalbantian said. Gamification software vendors say they can remove anything that signals the person's race or gender, and therefore remove opportunities for bias. Nalbantian said there is some truth to this. Another potential advantage of such software is that it may help companies consider candidates that have the right skills but have not held a particular type of job before. “When labor markets are tight and it's hard to keep good talent, being able to expand the potential labor pool to candidates that have adjacent skill sets and experience can be a very positive thing to have access to more people than you otherwise would,” he said. But, Nalbantian said, the software’s reliance on computer gaming skills could be problematic. “You can imagine that less privileged people and candidates who did not grow up playing computer games and did not have computers in their homes will be at a systematic disadvantage,” Nalbantian said. “You end up in effect self-selecting certain types of people with certain backgrounds that may relate to race and gender and that may create disparities.”  AI-generated pre-employment tests Most standard pre-employment tests used to be taken on paper, and all candidates took the same test. When such tests were first put online, everyone still got the same list of questions. Now artificial intelligence is being used to modify qualification tests on the fly, according to Ken Willner, partner at Paul Hastings. The AI decides what the next question ought to be based on how the candidate did on the previous questions. “Let's say the test is looking at your geometry skills,” he said. “An artificial-intelligence-aided test might give you a first question on geometry, and then based on how you did on that one, give you either a harder question or an easier question.”  Because of this, applicants get different questions based on how well they did. “That's a way perhaps of learning more precisely about someone's geometry skills, but it lends to difficulty in validating a test,” Willner added. “Everyone's not taking exactly the same test or at least answering exactly the same questions. That's something psychologists have been wrestling with somewhat, these tests that modify themselves based on the applicant.” A bank would want to make sure its vendor can show that it's measuring the same skill or the same ability, he said. Nalbantian is not bothered by the use of AI to assess proficiency levels, especially where jobs require specific knowledge and skill sets. “You may have an apprentice level, a professional level and a master level,” he said. If answers to the first few questions identify someone as being at a master level, then it makes sense to ask questions that will gauge where the person is within the spectrum of mastery of the skillset. Letting AI screen resumes To weed through thousands of job applicants, large companies typically use application tracking systems from companies like Lever or SAP SuccessFactors that can analyze large volumes of resumes and separate the job candidates that will get a call back from those who won’t. Where AI software relies on data on past decisions to screen current applicants, this could lead to perpetuating existing bias in an organization. For instance, if a company has hired people from Ivy League colleges in the past and AI software picks up on that, it could disadvantage people who for socioeconomic reasons were not able to attend an elite college. The employment data of a company that has hired mostly white men in the past will likely lack signals relevant to successful Black and female candidates. “Hypothetically speaking, you might find a word like ‘African’ that's being either correlated or negatively correlated with selection, and you sure don't want to have a word that is directly related to a protected characteristic as one of the criteria that is being used in making selections,” Willner said. “If there's bias in the criteria that you're using, then there could be bias in the results. But that's something that companies can and should address to the greatest extent possible by looking at what is being correlated and eliminating anything that shows any potential for bias.”  For instance, a company may find the word “baseball” correlates to being good at teamwork, but it may not have enough women in its sample for the word “softball” to come up. “If baseball is going to indicate someone's on the baseball team and therefore is good at teamwork, well, softball would too,” Willner said. “You want to look for things like that and you want to know what's in there and make sure this is not discriminatory and that it is relevant to the job. The use of AI for video interviews Most large companies, and some small ones, use video interview technology to conduct video interviews with job candidates. One popular provider, HireVue, is used by many banks including Bank of America, Goldman Sachs, JPMorgan, and Morgan Stanley. In 2019, the Electronic Privacy Information Center filed a complaint with the Federal Trade Commission alleging that HireVue uses facial recognition technology and proprietary algorithms to assess job candidates’ cognitive ability, psychological traits, emotional intelligence and social aptitudes. HireVue collects tens of thousands of data points from each video interview of a job candidate, including a candidate’s intonation, inflection, and emotions to predict each job candidate’s employability, EPIC said. EPIC also said HireVue does not give candidates access to their assessment scores or the training data, factors, logic, or techniques used to generate each algorithmic assessment. HireVue says it discontinued the facial analysis component of its algorithm in March 2020 after internal research showed that advances in natural language processing had increased the predictive power of language analysis. “Over time, we realized the minimal value provided by the visual analysis didn’t warrant continuing to incorporate it in the assessments or outweigh the potential concerns,” said Lindsey Zuloaga, chief data scientist at HireVue.