Exchangeability & Ignorability

Exchangeability is the 'apples-to-apples' rule for finding a true cause. It means the groups you compare must be so similar in all other relevant ways that it's as if they were randomly assigned to treatment. For example, if comparing library access to crime rates, we must ask if the neighborhoods also differ in wealth. If so, the groups aren't exchangeable.

Try this: If the groups being compared had swapped circumstances, would their outcomes have swapped too?

Exchangeability is the assumption that the outcome for one group would have been the same as the other group's outcome, had they received the same treatment. The term ignorability is often used interchangeably and means we can 'ignore' how people were assigned to groups, because the assignment was effectively random after accounting for key factors.

Think of a medical study comparing a new drug to a placebo. If doctors give the drug to sicker patients and the placebo to healthier ones, the groups are not exchangeable. You can't tell if a bad outcome is due to the drug or the patient's initial poor health. This is why randomized controlled trials (RCTs) are a gold standard: randomization creates exchangeable groups on average.

In real life, we rarely have randomization. An employer might notice that employees who take an optional leadership course get promoted more often. Are the groups (course-takers and non-takers) exchangeable? Probably not. The employees who sign up are likely more ambitious and motivated to begin with—factors that also lead to promotion. The ambitious employees are selecting into the treatment, a problem known as selection bias.

When we can't randomize, we try to make groups comparable by statistically adjusting for confounding variables like ambition, age, or experience. The goal is to achieve 'conditional exchangeability'—making the groups similar enough after we account for these other factors, so we can isolate the true effect of the course.

Exchangeability and ignorability are foundational concepts in modern causal inference that formalize the conditions needed to treat observational data as if it came from a randomized experiment [1, 2, 3].

• The concept is core to the potential outcomes framework, where the treatment assignment mechanism must be independent of the potential outcomes, a property Donald Rubin termed 'ignorability' [1]. (1974)
• In graphical models, exchangeability is achieved when there are no unblocked 'backdoor paths' between the treatment and outcome, a criterion formalized by Judea Pearl [2]. (2009)
• Modern epidemiology and social sciences use these principles to design observational studies and apply statistical adjustments to approximate the conditions of a randomized trial, aiming for 'conditional exchangeability' [3, 4]. [3] (2020)

The biggest limitation is that we can only adjust for measured confounders. If there are unmeasured factors that affect both the treatment and the outcome (e.g., a person's intrinsic motivation), exchangeability will be violated, and our causal estimate will be biased [2, 3]. This is often called 'residual confounding.' Furthermore, the assumption of exchangeability is fundamentally untestable from the data alone; we can only use domain knowledge and causal diagrams to argue for its plausibility [2].