Causal Analysis in Theory and Practice

Andrew Gelman wrote a follow up to his original post:

To follow up on yesterday's discussion, I wanted to go through a bunch of different issues involving graphical modeling and causal inference.

Contents:
– A practical issue: poststratification
– 3 kinds of graphs
– Minimal Pearl and Minimal Rubin
– Getting the most out of Minimal Pearl and Minimal Rubin
– Conceptual differences between Pearl's and Rubin's models
– Controlling for intermediate outcomes
– Statistical models are based on assumptions
– In defense of taste
– Argument from authority?
– How could these issues be resolved?
– Holes everywhere
– What I can contribute

Andrew Gelman (Columbia) recently wrote a blog post motivated by Judea Pearl's paper, "Myth, Confusion, and Science in Causal Analysis. " In response, Pearl writes:

Dear Andrew,

Thank you for your blog post dated July 5. I appreciate your genuine and respectful quest to explore the differences between the approaches that I and Don Rubin are taking to causal inference.

In general, I would be the first to rally behind your call for theoretical pluralism (e.g., "It make sense that other theoretical perspectives such as Pearl's could be useful too.") We know that one can prove a theorem in geometry by either geometrical or algebraic methods, depending on the problem and the perspective one prefers to take–only the very dogmatic would label one of the methods "unprincipled".

My article, "Myth, confusion and Science in Causal Analysis", is written with this dual perspective in mind, fully accommodating the graphical and potential-outcome conceptualizations as interchangeable, "A theorem in one approach is a theorem in another," I wrote.

However, when adherents of the one-perspective approach make claims that mathematically contradict those derived from the dual-perspective approach, one begins to wonder whether there is something more fundamental at play here.

(more…)

Tutorial
Judea Pearl will be presenting a tutorial at the JSM meeting (Washington, DC August 5, 2009 from 2-4pm) on "Causal Analysis in Statistics: A Gentle Introduction"

Additional information about the session may be obtained by clicking here.

Book Signing
Just before the tutorial at 12 noon, there will be a book-signing gathering at the Cambridge University Press booth, where J. Pearl will be signing copies of the 2nd Edition of Causality and will engage in gossip and debates about where causality is heading.

Ilya Shpitser and Judea Pearl recently presented an article at the UAI conference which offers a solution to the (counterfactual) problem of evaluating the Effect of Treatment on the Treated.

The article may be viewed here: http://ftp.cs.ucla.edu/pub/stat_ser/r349.pdf

Several of Professor Pearl's recent seminars on causality are now available for streaming online. Please follow the links below to view them.

• State of Causality in Economics (A Joint Seminar with UCLA, UCI, UCSD) (Requires RealPlayer)
• A Seminar given to Johns Hopkins University Bloomberg School of Public Health
• A Seminar given to USC Information Sciences Institute

Please feel free to post any comments or questions about these videos in the comments section.

A letter from Judea Pearl to the Editor of Statistics in Medicine:

Dear Editor,

I read with great interest Donald Rubin’s paper "The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials" (2007, 26(1):20-36) [1], as well as the Letter To The Editor by Ian Shrier (2008, 27(14):2740-2741) [2], and Author Reply by Don Rubin (2008, 27(14):2741-2742) [3].

Shrier’s Letter posed an important question which remains unanswered in Rubin’s reply. I here venture to answer this question and to clarify related issues concerning the interpretation of propensity scores (PS) and their role in causal inference.

Shrier’s question was whether, asymptotically, the use of PS methods as described by Rubin may actually increase, not decrease, bias over and above a crude, unadjusted comparison between treated and untreated subjects. The answer is: Yes, and the M-graph cited by Shrier (see also [4, 5]) provides an extreme such example; the crude estimate is bias-free, while PS methods introduce new bias.

This occurs when treatment is strongly ignorable to begin with and becomes non-ignorable at some levels of e. In other words, although treated and untreated units are balanced in each stratum of e, the balance only holds relative to the covariates measured; unobserved confounders may be highly unbalanced in each stratum of e, capable of producing significant bias. Moreover, such imbalance may be dormant in the crude estimate and awakened throughthe use of PS methods.

There are other features of PS methods that are worth emphasizing.

First, the propensity score e is a probabilistic, not a causal concept. Therefore, in the limit of very large sample, PS methods are bound to produce the same bias as straight stratification on the same set of measured covariates. They merely offer an effective way of approaching the asymptotic estimate which, due to the high dimensionality of X, is practically unattainable with straight stratification. Still, the asymptotic estimate is the same in both cases, and may or may not be biased, depending on the set of covariates chosen.

Second, the task of choosing a sufficient (i.e., bias-eliminating) set of covariates for PS analysis requires qualitative knowledge of the causal relationships among both observed and unobserved covariates. Given such knowledge, finding a sufficient set of covariates or deciding whether a sufficient set exists are two problems that can readily be solved by graphical methods [6, 7, 4].

Finally, experimental assessments of the bias-reducing potential of PS methods (such as those described in Rubin, 2007 [1]) can only be generalized to cases where the causal relationships among covariates, treatment, outcome and unobserved confounders are the same as in the experimental study. Thus, a study that proves bias reduction through the use of covariate set X does not justify the use of X in problems where the influence of unobserved confounders may be different.

In summary, the effectiveness of PS methods rests critically on the choice of covariates, X, and that choice cannot be left to guesswork; it requires that we understand, at least figuratively, what relationships may exist between observed and unobserved covariates and how the choice of the former can bring about strong ignorability or a reasonable approximation thereof.

Judea Pearl
UCLA

REFERENCES

1. Rubin D. The design versus the analysis of observational studies for causal effects: Parallels with the designof randomized trials. Statistics in Medicine 2007; 26:20–36.
2. Shrier I. Letter to the editor. Statistics in Medicine 2008; 27:2740–2741.
3. Rubin D. Author’s reply (to Ian Shrier’s Letter to the Editor). Statistics in Medicine 2008; 27:2741–2742.
4. Greenland S, Pearl J, Robins J. Causal diagrams for epidemiologic research. Epidemiology 1999; 10(1):37–48.
5. Greenland S. Quantifying biases in causal models: Classical confounding vs. collider-stratification bias.pidemiology 2003; 14:300–306.
6. Pearl J. Comment: Graphical models, causality, and intervention. Statistical Science 1993; 8(3):266–269.
7. Pearl J. Causality: Models, Reasoning, and Inference. Cambridge University Press: New York, 2000.

In response to an email discussion between Sander Greenland and Tyler VanderWeele concerning the semantics of causal dags, I have offered the following comment:

The phrase "what is causal about … " can be answered at three distinct levels of discussion: 1. Interpretation, 2. Construction and 3. Validation.

1. Interpretation

Causal dag is a model, and, like any other model, it is a symbolic object that acts as an oracle, that is, it produces answers to a set of queries of interest. In our case, the queries of interest are those concerning interventions and counterfactuals.

A distinction should be made here between "Causal Bayesian Networks" which are oracles of interventional queries only (see Causality Def. 1.3.1 pp. 23,) and functional or counterfactual dags, which are smarter oracles (given all the functions), capable of answering interventional as well as counterfactual queries, via Eq. (3.51), pp 98. For example, Fig. 1.6 (a) is a Causal Bayesian Network but not a functional dag of the process described.

2. Construction

What questions should investigators ask themselves during the construction of a causal dag, so as to minimize the likelihood of misspecification?

Given the judgmental nature of the assumptions embedded in the dag, this is a cognitive question, touching on the way scientists encode experience.

Three observations:

2.1 Scientists encode experience in the form of scientific laws, that is, counterfactual value-assignment processes. To determine the value x that variable X takes on, Nature is envisioned as examining the values s of some other variables, say S, and deciding according to some function x=f(s) what value X would be assigned. This is a more fundamental conception of causation than intervention, and it applies to non-manipulable variables as well, hence my favorite counter-slogan: "Of course causation without manipulation," or "Causation precedes manipulation."

2.2 To match the nature of scientific thought, the construction of causal dags is best done in counterfactual vocabulary. People judgment about statistical dependencies emanate from their judgment about cause-effect relationships, and people's judgments about causation emanate from counterfactual thinking; (otherwise, why would David Hume and David Lewis be tempted to define the former in term of the latter and not the other way around? see pp. 238-9). Accordingly, questions such as "What other factors determine X beside S?" or "Are the omitted factors determining X correlated with those determining Y?" can be quite meaningful to investigators and, hence, can be answered reliably, which explains why students find it easier to think in terms of "the existence of a hidden common parent of two or more nodes."

2.3 There is, I admit, some finger crossing in such judgment, as there is in any judgment, but the amount of guesswork is much much less than in the highly respected "Let us assume strong ignorability" which no mortal understands, except through translation into "no correlated hidden factors."

3. Validation

Given a causal dag, are its predictions compatible with the set of observations and experiments that one can perform?. The three conditions of Def. 1.3.1 pp. 23, are sufficient for guaranteeing that ALL observational and interventional queries be answered correctly. What does it mean? It means that, once conditions i-iii are satisfied, we can predict the effects of any policy, atomic as well as compound, deterministic as well as stochastic, that can be specified in terms of direct changes onto a given set of variables in the study. (By this we exclude unanticipated side-effects).In reality, students who construct dags do not think interventions, nor is it reasonable to assume that all interventional distributions would be available to us (as is assumed in Def 1.3.1). Still, it is healthy to have a sufficient set of empirical validation criteria such as Def 1.3.1.

The new edition will (1) provide technical corrections, updates, and clarifications to all ten chapters in the original book; (2) add summaries of new developments at the end of each chapter; (3) elucidate subtle issues that readers have found perplexing in a new chapter.

Information about the upcoming release, including an updated table of contents, may be found here: http://bayes.cs.ucla.edu/BOOK-09/causality2-excerpts.htm.

We welcome your comments.

Consider a Markovian model [tex]$G$[/tex] in which [tex]$T$[/tex] stands for the set of parents of [tex]$X$[/tex].  From [tex]{em Causality}[/tex], Eq.~(3.13), we know that the causal effect of [tex]$X$[/tex] on [tex]$Y$[/tex] is given by

[tex]begin{equation} P(y|hat{x}) = sum_{t in T} P(y|x,t) P(t) %% eq 1  label{ch11-eq-a} end{equation}[/tex] (1).

Now assume some members of [tex]$T$[/tex] are unobserved, and we seek another set [tex]$Z$[/tex] of observed variables, to replace [tex]$T$[/tex] so that

[tex]begin{equation} P(y|hat{x}) = sum_{z in Z} P(y|x,Z) P(z) %% eq 2  label{ch11-eq-b} end{equation}[/tex] (2).

It is easily verified that (2) follow from (1) if [tex]$Z$[/tex] satisfies:

 .

Indeed, conditioning on [tex]$Z$[/tex], ([tex]$i$[/tex]) permits us to rewrite (1) as [tex][ P(y|hat{x}) = sum_{t} P(t) sum_z P(y|z,x) P(z|t,x) ][/tex] and ([tex]$ii$[/tex]) further yields [tex]$P(z|t,x)=P(z|t)$[/tex] from which (2) follows. It is now a purely graphical exercize to prove that the back-door criterion implies ([tex]$i$[/tex]) and ([tex]$ii$[/tex]). Indeed, ([tex]$ii$[/tex]) follows directly from the fact that [tex]$Z$[/tex] consists of nondescendants of [tex]$X$[/tex], while the blockage of all back-door path by [tex]$Z$[/tex] implies , hence ([tex]$i$[/tex]). This follows from observing that any path from [tex]$Y$[/tex] to [tex]$T$[/tex] in [tex]$G$[/tex] that is unblocked by [tex]${X,Z}$[/tex] can be extended to a back-door path from [tex]$Y$[/tex] to [tex]$X$[/tex], unblocked by [tex]$Z$[/tex].