Can you rely on AI? Use our checklist to avoid the pitfalls
In a world of deepfakes, hallucinations, and bias, we offer practical guidance on how you can trust your AI systems....
Audio available
Artificial Intelligence
How ‘causal’ AI can improve your decision-making
Many companies are rushing to incorporate AI into their business models without being able to accurately gauge its benefits. Applying the principles of causal inference takes away the guesswork.
The November 2022 release of ChatGPT changed the world, and businesses have been struggling to adapt ever since. Despite this transformational shift, “traditional” AI like ChatGPT has its limitations – it gives us the how but not the why. That’s the reason tech giants Amazon, Meta, and Netflix – among many others – are hiring hundreds of experts in something that does: “causal” AI. But what is it?
Causal AI provides reliable, interpretable, and precise answers. It can accurately measure the ROI of an AI investment and much more. Precision is key for data-driven decision-making, as a lack of accuracy will likely translate into significant financial loss. As Julie Novak, a former senior data scientist at Netflix, said of the company’s working methods: “It’s not someone going on a hunch thinking: ‘Oh, this sort of marketing or advertising works best.’ Everything is rigorously tested.”
Let’s explore the differences between traditional AI and causal AI.
Traditional AI excels at predictive modeling through predictive inference, which involves guessing the value of something based on observed information. For instance, seeing people in shorts and sunglasses is a good predictor of warm weather. But the predictors – what people are wearing – are not causes. It is not sunny because people are wearing shorts.
Predictive inference is valuable for numerous applications, including:
- Spam filtering: Predicting the likelihood that an incoming email is spam based on its content.
- Medical diagnostics: Detecting tumors in MRI images.
- Fraud detection: Flagging potential fraudulent banking activities.
- Churn analysis: Identifying which customers are likely to leave based on historical data and behavior patterns.
- Sales forecasting: Predicting future sales using historical data, market trends, and economic indicators.
As predictive inference relies on correlations, these models can’t pinpoint the cause and measure their effects. “In most situations, businesses don’t care purely about prediction,” says data scientist Matheus Facure, the author of Causal Inference in Python. “They care about making decisions that bring more customers, increase conversion, decrease churn, increase profitability, and cut costs.”
What is causal AI, and how does it work?
Causal AI combines two fields: one part AI and nine parts causal inference. One simple example of causal analysis is A/B testing – a randomized experiment where you show different versions of a product or advertisement to customers and identify the best version based on a range of metrics.
Causal inference is the scientific method for measuring cause and effect. It’s about understanding the impact of one variable on another or identifying the cause of an outcome.
Here are examples where it is useful:
- Return on investment (ROI): Determining the effect of spending money on specific actions.
- Return on ad spend (ROAS): Understanding the effect of advertising on outcomes, accounting for factors like seasonality or competitor behavior to avoid biased estimations.
- Policy evaluation: Measuring the impact of policy changes, such as new work-from-home rules or executive compensation adjustments on performance.
- Identifying drivers: Understanding whether changes in sales are due to external factors (like economic conditions or the weather) or internal decisions (like marketing efforts or strategic choices).
These models allow us to interpret the result and the process, which is valuable for data-driven decision-making.
“Imagine you sell football jerseys, and you increase ad spending around big tournaments. If you simply observe that sales go up when you spend more on ads, you risk overestimating your intervention’s effectiveness while failing to account for the tournament’s influence on sales.”
Don’t use predictive inference to answer causal questions
Causal inference is different from predictive inference. When considering causality, we’re not interested in merely predicting the value of an outcome based on observed information. Instead, we want to understand what would happen to the outcome if we changed or intervened in a specific factor whose effect is of interest. For example, how would sales change if we increased ad spending?
Imagine you sell football jerseys, and you increase ad spending around big tournaments. If you simply observe that sales go up when you spend more on ads, you risk overestimating your intervention’s effectiveness while failing to account for the tournament’s influence on sales. This could lead to an overspend on ads during the off-season – expecting similar returns but being disappointed. Predictive inference relies on correlation. However, if you know correlation does not imply causation, you should recognize that traditional machine learning isn’t suitable for causal analysis.
You can use causal inference to measure the business impact of deploying a recommendation system or any other AI tool.
Why you should use causal inference with predictive AI
While predictive and causal inference serve different roles, they can work together to enhance decision-making. Dima Goldenberg, Senior Machine Learning Manager at online booking agency Booking.com, illustrates this integrated framework well. Like many online platforms, the company uses recommendation algorithms to enhance user experience. Predictive AI is ideal for anticipating what a user is most likely to be interested in based on past behavior and user characteristics. However, once you have a powerful recommendation model, another crucial question arises: What is the impact of this recommendation on conversion rates or sales?
You can use causal inference to measure the business impact of deploying a recommendation system or any other AI tool. For example, you could use A/B testing to compare the conversion or value of the basket with or without the recommendation system. The same logic allows you to decide whether to adapt, scale up, or discontinue it. It is also central for optimization, which allows you to compare different variations of AI products using A/B testing.
Companies rarely report the results of their experiments publicly. However, the Delhi-based company VWO, which specializes in A/B testing, did share some success stories. Ubisoft, a French maker of video games including the Just Dance and Assassin’s Creed franchises, wanted to improve the conversion rate of its “buy now” page for its For Honor brand. Having collected data using click maps, scroll maps, heat maps, and surveys, the company concluded that the buying process was too unwieldy. It redesigned the product page to reduce the amount of scrolling needed and carried out A/B testing for three months. Ubisoft found that the variation increased conversions from 38% to 50%.
What is Causal AI?
We have seen that predictive inference benefits from the use of causal inference. Causal AI is the opposite relationship. In causal AI, we use the capacity of predictive AI to detect patterns efficiently and solve technical issues. In causal AI, predictive AI for both outcomes and interventions works alongside causal analysis.
First, predictive AI identifies key background factors (like past purchases, income, or age) that influence both intervention uptake and outcomes. Then, causal analysis uses this information to balance treated and control groups – ensuring comparability – and accurately estimate the intervention’s effect. This two-step approach bridges predictive modeling and causal inference to yield more reliable conclusions.
One of the most valuable tools in causal AI is the “causal forest.” Traditionally, when running a randomized experiment in medicine or in businesses with A/B testing, the goal is to measure the average difference between two groups. But what might work for one person might not work for another, and the average might hide this heterogeneity. Maybe your new marketing campaign benefited the younger audience, but the older audience didn’t like it. Predictive AI is excellent at detecting these patterns and so can be used as a second step after a randomized experiment to identify what we call “heterogeneous treatment effects.” In other words, pinning down how different groups reacted to the treatment and allowing the personalization of further business decisions without harming subgroups.
A company’s A/B testing might reveal that personalized recommendation algorithms increased the average holiday spend by 12%. To understand which types of users benefited most from the intervention, the company could analyze heterogeneous treatment effects using a combination of subgroup analysis and predictive AI. By examining how user characteristics – such as age, gender, or income – interact with the treatment, the company can identify subgroups for whom the recommendation algorithm was particularly effective (or ineffective), allowing for more targeted and optimized personalization strategies.
Business leaders must re-examine their current analytics approaches.
Decision-making in the age of AI
Understanding the difference between predictive and causal inference is a strategic necessity. By grasping causality, businesses can optimize decisions and allocate resources more effectively. Ultimately, they can attain better outcomes by accurately measuring the impact of their investment in chatbots or predictive AI.
Business leaders must re-examine their current analytics approaches. It’s time to invest in the right tools, build teams with expertise in causal inference, and foster a culture that values the understanding of cause and effect. By integrating causal methods into your decision-making processes, you’ll avoid costly mistakes and unlock opportunities for growth and innovation.
Authors
Related
What do you know about cybersecurity risks when scaling industrial AI?
Companies are rushing to embrace the game-changing opportunities to improve operations and scale that AI offers; yet many are unaware of the pitfalls. Test your knowledge of the risks here – and...
How businesses should navigate psychological profiling
In an era defined by advanced AI and behavioral data, Apple’s 'Evil Steve' test exemplifies a proactive approach to ethical decision-making and responsible data governance....
AI maturity in financial services: lessons from the most successful firms
The race to harness artificial intelligence is reshaping the financial services landscape, with industry leaders demonstrating how strategic AI implementation drives competitive advantage. IMD's AI Maturity Index reveals the winning formula. ...
Learn Brain Circuits
Join us for daily exercises focusing on issues from team building to developing an actionable sustainability plan to personal development. Go on - they only take five minutes.
Explore Leadership
What makes a great leader? Do you need charisma? How do you inspire your team? Our experts offer actionable insights through first-person narratives, behind-the-scenes interviews and The Help Desk.
Join Membership
Log in here to join in the conversation with the I by IMD community. Your subscription grants you access to the quarterly magazine plus daily articles, videos, podcasts and learning exercises.