What AI can teach us about human bias in decision making

The debate between meritocracy and affirmative action has long simmered, and the arguments on both sides are familiar. But quickly to recap, let’s look at an exchange between two people, each on either side of the argument. 

Vivek Ramaswamy is a self-made multimillionaire who founded a biotech company at the age of 28. He is a celebrity in conservative circles, having been a regular guest on Fox News and last year published Woke, Inc: Inside the Social Justice Scam, which takes corporations to task for — as he sees it — overstepping the mark in becoming involved in moral issues best left to society.  

Recently, he recalled a debate with Kenneth Frazier, the former chief executive of the pharmaceuticals company Merck. Frazier had said in the wake of the police killing of African-American George Floyd: “There are, in fact, barriers that are faced by African Americans … We still have customs. We still have beliefs. We still have policies. We have practices that lead to inequity.” Therefore, he argued, “Businesses have to use every instrument at their disposal to reduce these barriers that existed [sic].” In short, he believed that companies needed to actively tilt the balance in favor of the disadvantaged.   

Yet Ramaswamy was skeptical. For him, racial diversity was just another fashionable social cause, such as (as he saw it) gender or climate change. Companies embraced these causes because they were on trend, he asserted. A pharmaceutical company should focus instead on serving patients first, and that means, Ramaswamy wrote: “We don’t care if the scientist who discovers a cure to COVID-19 is white or black, or a man or a woman.” 

If only things were that simple. We are all human. We have limited knowledge. We may understand others well if they have had similar life experiences to ours. Human psychology tricks us, too. It makes us want to be around people who are like us — people with whom we can sympathize. That is all very innocent until you consider the cost to society. 

The long history of gender bias in medicine 

Take medicine. Men and women have different ways of reacting to illness. Hippocrates noted this over 2,000 years ago. Women with heart disease, for instance, often experience different symptoms than men with the same disease. Women might feel pain in the abdomen rather than the left arm.  

Despite these distinctions, many modern-day clinical studies are stubbornly influenced by gender bias. Although we know that women’s bodies metabolize medicines differently, only 41.9% of participants in cardiovascular research are female. A shocking fact is that 51% of cancer patients are women, whereas only 41% of cancer trial participants are women. 

Women make up 60% of individuals who suffer from mental illness and yet, even in today’s modern medical environment, only 42% of participants in psychiatric drug trials are women. Changes in a woman’s hormones, such as those that occur during puberty, pregnancy, and menopause, have been linked to an increased risk of depression. 

This means that a woman is likely to become depressed during these times. As an instance, my own grandmother committed suicide after battling postpartum depression for months, if not years.

Aside from their reproductive organs, women are generally regarded as comparable to men, albeit often smaller in size. Gatekeepers, including doctors, surgeons, policy makers, and medical researchers, are predominantly male. Their viewpoints have become entrenched in the way we approach medicine. 

The reality is that no matter how compassionate and intelligent a white man may be, it is impossible for him to experience the human condition of a black woman firsthand. There are things about a woman’s wellbeing that men will never realize until they are told.  

We can see, therefore, that the medical world is a good, real-world example of what happens when you rely on one side of the argument to drive decisions and outcomes. This is why we need to actively tilt the gender balance to improve the overall system. A homogeneous group will have a hard time empathizing with those who do not share their life experiences. If you do not actively pursue diversity, the status quo will remain. That is just how decisions are made. The way we see the world is shaped by our personal experiences. Observable traits, such as race, ethnicity, gender, and age, are the starting point of diversity of thought.  

Evidence of how diversity works from the world of AI 

Now, you may ask, does cognitive diversity really deliver superior performance, and isn’t meritocracy a better way to go? Numerous studies on innovation have suggested it actually does. However, the most compelling explanation can be found in the field of artificial intelligence and the algorithms associated with it.  

Algorithms can be best understood as predictive machines. They are used to make decisions or recommendations. The question they always answer is: what should be done next? And they do so by sifting through data to find patterns. The more the data, the better the pattern recognition. 

Machine algorithms already do things that would take too much time or be too hard for humans to accomplish. They predict how we click, buy, lie, or die. Companies use them to offer a discount, recommend a product, show an ad, inspect for flaws, and approve a loan. Credit card companies can detect fraudulent transactions in real time, while insurers can predict who is likely to make a claim. 

Despite its abundance, data is not free. Good quality data is hard to come by. Therefore, companies need to figure out how to best engineer the most potent algorithm given what they have. An approach called the ensemble method deserves mention here. It essentially harnesses cognitive diversity in the world of AI. 

The ensemble method works by dividing an existing dataset into small samples, which can then independently train different algorithms at the same time. In these smaller sample sets, these algorithms tend to be weaker. They make more errors in their predictions on their own. The wonderful thing is that you can aggregate their results. 

The collective insight, or total wisdom of these algorithms is far superior to that of a single algorithm. In fact, it is better than a single algorithm that is trained by all the data in one go. In other words, the collective power of the weak is mightier than that of the strongest individual.  

There are two key reasons for the power of this collective: firstly, algorithms tend to make different types of errors, and secondly, they learn from each other and improve as they go along. This is what computer scientists call co-evolution. The main idea is that a diverse set of algorithms will have superior performance. This is because of their complementary strengths and weaknesses. And they shine especially when making prediction in the new, unseen data.  

I describe all this because of what it tells us in relation to the benefits and uses of diversity, and not some narrow focus on meritocratic outcomes. The ensemble method in AI is what we need to accept in the real world. Each person comes with their unique perspective. When facing a complex problem, or in a novel situation, no one will have all the relevant insights — and here lies the problem of smart people who look alike. They think in the same way. They mirror each other’s perspective. They then confirm their own biases. They do not even know what they do not know.  

Just as with machine learning, humans need to actively seek out diverse perspectives of people who are wired differently because of their alternative life experiences. Only then can we build a better, more accurate picture of the truth. Furthermore, we can make better decisions for society when we understand things better. 

Medicine needs diversity. Now imagine how much diversity is needed in other areas such as developing new products, designing engineering projects, and providing legal advice.  

How far companies have come in harnessing diversity  

At IMD’s Center for Future Readiness, we have been tracking how many women are in executive positions, the diversity of leadership teams, and the backgrounds of CEOs. We also look at their position on environmental, social, and corporate governance (ESG). We run text analyses on what is written on companies in these areas.  

The situation in the clinical trial sector, as we saw previously, is far from ideal. However, if you look at the overall picture and compare it with other industries, it is not the worst. 

We used big data to examine how much attention different industries pay to gender equality and racial justice. We reviewed reports from the last 10 years from The Wall Street Journal, New York Times, Financial Times, and other business news sources. We also looked at all the corporate press releases from the same period. We fed all the data into an algorithm. The algorithm considered what different companies represent and how well the business community understands these companies. We put all this information together to get a sense of how things are going in the industry. This is not perfect, but it helps us understand what is happening on a large scale. 

We noticed that pharmaceutical and technology companies pay attention to diversity issues. In the first graph, the Y-axis shows how much they focus on this issue, while the X-axis shows how much a company focuses on taking advantage of opportunities (exploit) versus exploring new areas (explore). A company always needs to take advantage of opportunities that are currently available. However, if it only focuses on the short term, the future will be difficult. It also needs to explore new opportunities, such as research and development, invention, and creative thinking.  

Explore and exploit are not mutually exclusive choices. Companies must have both. Still, in industries that are turbulent and changing quickly, companies tend to explore more. They also embrace diversity in gender, race, and sexual orientation. Presumably, that is because they need different kinds of thinking to challenge themselves.  

In the second graph, we plot how different industries learn. The X-axis shows how oriented an industry is towards learning. Again, we see that pharmaceutical and technology industries are more learning-oriented than other industries. They are more likely than others to pay more attention to diversity.

All this does not mean a lack of diversity will immediately result in poor performance. Different industries have different dynamics, while companies also have different traits. However, one thing is clear: when the market changes, successful companies change their outlook to adapt. 

In a way, Ramaswamy is both right and wrong. The government should not need to tell business people who to hire, yet businesses themselves cannot self-correct to yield optimal results for society, and that’s because businesses are run by people. People have their own biases and, without intervention, the system would self-perpetuate.  

It also matters whether a scientist or an engineer is black or white, male or female. If everyone looks the same, the team cannot solve problems effectively.