Understanding Causality: The Gold Standard in Research Methods
Written on
Chapter 1: The Importance of Causal Relationships
Establishing causal relationships is a vital aspect of effective communication and decision-making. It allows us to confidently assert reasons behind changes, such as, "Our sales increased because we redesigned the website." Accurately identifying these causal links is essential not only in business for enhancing operational efficiency but also in governmental policy-making to ensure taxpayer funds are utilized effectively. This article series will delve into four statistical methods that lend scientific validity to causal assertions.
The cornerstone of causality is rigorous evidence. The four methodologies we will examine include:
- Randomized experiments
- Instrumental variables
- Regression discontinuity
- Difference-in-differences
In this initial installment, we will focus on randomized experiments, regarded as the gold standard in scientific inquiry.
Section 1.1: Correlation vs. Causation
It is often stated that correlation does not imply causation. Just because two events happen simultaneously does not mean one causes the other. A classic example is the correlation between the number of people who drown in pools and the number of Nicolas Cage films released. This correlation seems absurd; how could the release of films possibly affect drowning rates?
To explore the true nature of causality, we need precise statistical tools. The most straightforward and effective method is through randomized experiments.
Subsection 1.1.1: Introducing the Potential Outcome Model
Before we discuss randomized experiments, it’s important to understand the Potential Outcome Model, a framework commonly used to analyze causality. This model often applies medical research terminology, as its early applications focused on evaluating whether treatments improved patients' health.
In this model, two key variables are identified:
- Cᵢ = {0, 1}, indicating whether individual i received the treatment.
- Yᵢ, representing the outcome for individual i.
The treatment (C = 1) can refer to anything from administering a medication to displaying a new website layout. Regardless of context, the same analytical framework is applicable.
The outcomes for an individual can be defined as:
- Y₁ᵢ: the result if individual i received the treatment (Cᵢ = 1)
- Y₀ᵢ: the result if individual i did not receive the treatment (Cᵢ = 0)
Since it is impossible for one individual to experience both outcomes simultaneously, we rely on a hypothetical scenario to determine the treatment effect:
TEᵢ = Y₁ᵢ - Y₀ᵢ
For instance, if a user spends 50 seconds on a new website layout (Y₁ᵢ) and only 30 seconds on the old one (Y₀ᵢ), the treatment effect would be 20 seconds.
To ascertain the average impact across multiple individuals, we calculate the Average Treatment Effect (ATE):
ATE = E[Y₁ᵢ - Y₀ᵢ]
However, ATE is often not our primary concern. Instead, we are typically interested in the Average Treatment Effect on the Treated (ATET), which focuses on the individuals who actually received the treatment:
ATET = E[Y₁ᵢ - Y₀ᵢ | Cᵢ = 1]
Unfortunately, calculating ATET is complicated by the challenge of determining counterfactuals.
Section 1.2: The Pitfalls of Naive Comparison
What if we simply compared average outcomes between treated and untreated groups? This naive comparison could be misleading. For example, if treated patients have an average of 90 virus cells per 1cm³ compared to 140 for untreated ones, it does not necessarily demonstrate the drug’s causal effect.
By introducing the counterfactual for treated patients, we can see that this naive comparison includes selection bias, complicating our understanding of causality.
Chapter 2: The Efficacy of Randomized Experiments
The solution to overcoming selection bias is the randomized experiment. This method involves randomly assigning participants into two groups: one receiving the treatment and the other serving as a control.
The rationale is that with a sufficiently large sample and proper randomization, all extraneous variables will be evenly distributed between the groups. Therefore, any difference in outcomes can be attributed to the treatment itself.
In a properly conducted randomized experiment, the difference in average outcomes indeed reflects the causal impact of the treatment.
The first video titled "Identification Strategies, Part 1: How Economists Establish Causality" provides an overview of various strategies for establishing causal relationships in economics.
However, randomized experiments aren't always feasible or ethical, particularly in social sciences. For instance, can we randomly impose tax reforms on citizens to observe their effects? Ethical considerations often dictate that alternative methods must be used to estimate causal impacts.
Next, we will examine the instrumental variables approach.
Thanks for reading!
To stay updated on the rapidly evolving fields of machine learning and AI, consider subscribing to my newsletter, AI Pulse. For consulting inquiries, feel free to reach out or book a session with me.
You can also explore my other articles. Unsure where to start? Here are some recommendations:
The second video titled "Experiments (establishing causation)" discusses the fundamental principles behind conducting experiments to determine causality.