Causal Analysis in Theory and Practice

In a recent survey on Instrumental Variables (link), Guido Imbens fleshes out the reasons why some economists “have not felt that graphical models have much to offer them.”

His main point is: “In observational studies in social science, both these assumptions [exogeneity and exclusion] tend to be controversial. In this relatively simple setting [3-variable IV setting] I do not see the causal graphs as adding much to either the understanding of the problem, or to the analyses.” [page 377]

What Imbens leaves unclear is whether graph-avoiding economists limit themselves to “relatively simple settings” because, lacking graphs, they cannot handle more than 3 variables, or do they refrain from using graphs to prevent those “controversial assumptions” from becoming transparent, hence amenable to scientific discussion and resolution.

When students and readers ask me how I respond to people of Imbens’s persuasion who see no use in tools they vow to avoid, I direct them to the post “The deconstruction of paradoxes in epidemiology”, in which Miquel Porta describes the “revolution” that causal graphs have spawned in epidemiology. Porta observes: “I think the “revolution — or should we just call it a renewal”? — is deeply changing how epidemiological and clinical research is conceived, how causal inferences are made, and how we assess the validity and relevance of epidemiological findings.”

So, what is it about epidemiologists that drives them to seek the light of new tools, while economists (at least those in Imbens’s camp) seek comfort in partial blindness, while missing out on the causal revolution? Can economists do in their heads what epidemiologists observe in their graphs? Can they, for instance, identify the testable implications of their own assumptions? Can they decide whether the IV assumptions (i.e., exogeneity and exclusion) are satisfied in their own models of reality? Of course the can’t; such decisions are intractable to the graph-less mind. (I have challenged them repeatedly to these tasks, to the sound of a pin-drop silence)

Or, are problems in economics different from those in epidemiology? I have examined the structure of typical problems in the two fields, the number of variables involved, the types of data available, and the nature of the research questions. The problems are strikingly similar.

I have only one explanation for the difference: Culture.

The arrow-phobic culture started twenty years ago, when Imbens and Rubin (1995) decided that graphs “can easily lull the researcher into a false sense of confidence in the resulting causal conclusions,” and Paul Rosenbaum (1995) echoed with “No basis is given for believing” […] “that a certain mathematical operation, namely this wiping out of equations and fixing of variables, predicts a certain physical reality” [ See discussions here. ]

Lingering symptoms of this phobia are still stifling research in the 2nd decade of our century, yet are tolerated as scientific options. As Andrew Gelman put it last month: “I do think it is possible for a forward-looking statistician to do causal inference in the 21st century without understanding graphical models.” (link)

I believe the most insightful diagnosis of the phenomenon is given by Larry Wasserman:
“It is my impression that the “graph people” have studied the Rubin approach carefully while the reverse is not true.” (link)

A new review of my book *Causality* (Pearl, 2009) has appeared in the Journal of Structural Equation Modeling (SEM), authored by Stephen West and Tobias Koch (W-K). See http://bayes.cs.ucla.edu/BOOK-2K/west-koch-review2014.pdf

I find the main body of the review quite informative, and I thank the reviewers for taking the time to give SEM readers an accurate summary of each chapter, as well as a lucid description of the key ideas that tie the chapters together. However, when it comes to accepting the logical conclusions of the book, the reviewers seem reluctant, and tend to cling to traditions that lack the language, tools and unifying perspective to benefit from the chapters reviewed.

The reluctance culminates in the following paragraph:
“We value Pearl’s framework and his efforts to show that other frameworks can be translated into his approach. Nevertheless we believe that there is much to be gained by also considering the other major approaches to causal inference.”

W-K seem to value my “efforts” toward unification, but not the unification itself, and we are not told whether they doubt the validity of the unification, or whether they doubt its merits.
Or do they accept the merits and still see “much to be gained” by pre-unification traditions? If so, what is it that can be gained by those traditions and why can’t these gains be achieved within the unified framework presented in *Causality*?

To read more, click here.

Simpson’s paradox must have an unbounded longevity, partly because traditional statisticians, so it seems, are still refusing to accept the fact that the paradox is causal, not statistical (link to R-414).

This was demonstrated recently in an April discussion on Gelman’s blog where the paradox was portrayed again as one of those typical cases where conditional associations are different from marginal associations. Strangely, only one or two discussants dared call: “Wait a minute! This is not what the paradox is about!” — to little avail.

To watch the discussion more closely, click http://andrewgelman.com/2014/04/08/understanding-simpsons-paradox-using-graph/ .

A recent discussion that might be of interest to readers took place on SEMNET, a Structural Equation Modeling Discussion Group, which appeals primarily to traditional SEM researchers who, generally speaking, are somewhat bewildered by the recent fuss about modern causal analysis. This particular discussion focused on “causal mediation”.

1.
An SEMNET user, Emil Coman, asked (my paraphrasing):
“Who needs causal mediation (CM)?”
All it gives us is: (a) the ability to cope with confounders of the M—>Y relation and (b) the ability to handle interactions. Both (a) and (b) are SEM-fixable; (a) by adjusting for those confounders and (b) by using Bengt Muthen’s software (Mplus), whenever we suspect interactions.

To continue, click here.

A lively discussion flared up early this month on Andrew Gelman’s blog (garnering 114 comments!) which should be of some interest to readers of this blog.

The discussion started by a quote from George Box (1979) on the advantages of model-based approaches, and drifted into related topics such as

(1) What is a model-based approach,

(2) Whether mainstream statistics encourages this approach,

(3) Whether statistics textbooks and education have given face to reality,

(4) Whether a practicing statistician should invest time learning causal modeling,

or wait till it “proves itself” in the real messy world?

I share highlights of this discussion here, because I believe many readers have faced similar disputations and misunderstandings in conversations with pre-causal statisticians.

To read more, click here.

UCLA Department of Statistics Seminar Series

Thu, 10/24/2013, 12:30 PM—1:30 PM
4660 Geology Bldg.

Judea Pearl and Joakim Ekstrom
Statistics
UCLA

A Conversation on Statistical Methodology, with Judea Pearl
http://statistics.ucla.edu/seminars/2013-10-24/12:30pm/4660-geology

Join us for a conversation on statistical methodology, and in particular the theory of causal inference. In this ‘Socratic dialogue’-styled conversation, recent A.M. Turing award winner Judea Pearl will discuss his views on statistical methodology with conversational partner Joakim Ekstrom. The conversation will start at R.A. Fisher’s randomization procedure for isolation of contributors to systematic variation, and then continue discussing the methodology of Judea Pearl for isolation and identification of causal factors in data obtained from sources other than perfectly randomized experiments.

In the conversation, there will be plenty of opportunity for attendees to ask questions, explore alternatives and raise objections, especially regarding ways of introducing causal inference in statistics education.

Judea Pearl is a Professor at UCLA Computer Science and Statistics, and has contributed greatly to the theory of causal inference. Joakim Ekstrom is a post-doctoral research scholar at UCLA Statistics, seminar co-organizer, and an expert on the theory and history of statistics.

Larry Wasserman posted the following comments on his “normal-deviate” blog:
http://normaldeviate.wordpress.com/2013/08/09/the-jsm-minimaxity-and-the-language-police/

I am back from the JSM (http://www.amstat.org/meetings/jsm/2013/). For those who don’t know, the JSM is the largest statistical meeting in the world. This year there were nearly 6,000 people.

*******skipping *******
On Tuesday, I went to Judea Pearl’s medallion lecture, with discussions by Jamie Robins and Eric Tchetgen Tchetgen. Judea gave an unusual talk, mixing philosophy, metaphors (eagles and snakes can’t build microscopes) and math. Judea likes to argue that graphical models/structural equation models are the best way to view causation. Jamie and Eric argued that graphs can hide certain assumptions and that counterfactuals need to be used in addition to graphs.
***********more *********

J. Pearl:

I posted the following reply:

Larry,

Your note about my Medallion Lecture (at JSM 2013) may create the impression that I am against the use of counterfactuals.

This is not the case.

1. I repeatedly say that counterfactuals are the building blocks of rational behavior and scientific thoughts.
see: http://ftp.cs.ucla.edu/pub/stat_ser/R269.pdf
http://ftp.cs.ucla.edu/pub/stat_ser/r360.pdf

2. I showed that ALL counterfactuals can be encoded parsimoniously in one structural equation model, and can be read easily from any such model.
see: http://ftp.cs.ucla.edu/pub/stat_ser/r370.pdf

3. I showed how the graphical-counterfactual symbiosis can work to unleash the merits of both. And I emphasized that mediation analysis would still be in its infancy if it were not for the algebra of counterfactuals (as it emerges from structural semantics.)

4. I am aware of voiced concerns about graphs hiding assumptions, but I prefer to express these concerns in terms of “hiding opportunities”, rather than “hiding assumptions” because the latter is unnecessarily alarming.

A good analogy would be Dawid’s notation X||Y for independence among variables, which states that every event of the form X = x_i is independent of every event of the form Y=y_j. There may therefore be hundreds of assumptions conveyed by the innocent and common statement X||Y.

Is this a case of hiding assumptions?
I do not believe so.

Now imagine that we are not willing to defend the assumption “X = x_k is independent of Y=y_m” for some specific k and m. The notation forces us to write “variable X is not independent of variable Y” thus hiding all the (i,j) pairs for which the independence is defensible. This is a loss of opportunity, not a hiding of assumptions, because refraining from assuming independence is a more conservative strategy; it prevents unwarranted conclusions from being drawn.

Thanks for commenting on my lecture.

Judea Pearl Writes:

Last week I attended a causal inference symposium at the University of Michigan, and had a very lively discussion with James Heckman (Chicago, economics) on causal reasoning in econometrics, statistics and computer science. Video and slides of the two lectures can be watched here: http://www.psc.isr.umich.edu/pubs/video-tapes.html

In the QA session (not in the video), I described the problems of transportability and external validity, and their solutions according to:
http://ftp.cs.ucla.edu/pub/stat_ser/r372.pdf
http://ftp.cs.ucla.edu/pub/stat_ser/r390.pdf

Heckman asked: What makes this problem different from the one that economists solve routinely — when they find a new distribution that differs from the one they estimated, they simply re-estimate the parameters by which the two differ and keep those on which they agree.

My answer stressed three facts that should be kept in mind when dealing with “transporatability”:
1. We cannot speak here about differing “distributions” because transportability is a causal, not statistical problem. In other words, what needs to be re-estimated depends not on the two “distributions” but on the causal story behind the distributions. (This is shown vividly in Example 2 of R-372).

2. We are now dealing with the task of transporting “experimental findings” (e.g., causal effects), not distributions, from a place where they are available to a place where they are not estimable.

3. We cannot even speak about re-estimating “parameters” because the problem is entirely non-parametric.

More comments on audience questions will follow.

A recent discussion on Andrew Gelman’s blog has touched on some interesting points concerning the teaching of causality in econometric and statistics classes (link here). I responded to some of the discussants and, below, I share my replies with readers of this blog.

1. Andrew Gelman asked why the review in http://ftp.cs.ucla.edu/pub/stat_ser/r395.pdf is critical of econometrics, “I thought that causality was central to econometrics; see, for example, Angrist and Pischke’s book .”

Judea Pearl replies:
Causality is indeed central to econometrics. Our survey of econometric textbooks http://ftp.cs.ucla.edu/pub/stat_ser/r395.pdf is critical of econometric education today, not of econometric methodology proper. Econometric models, from the time of Haavelmo (1943), have been and remained causal (see http://ftp.cs.ucla.edu/pub/stat_ser/r391.pdf ) despite two attempted hijacking, first by regressionists, and second by “quasi-experimentalists,” like Angrist and Paschke (AP). The six textbooks we reviewed reflect a painful recovery from the regressionist assault which more or less disappeared from serious econometric research, but is still obfuscating authors of econometric textbooks.

As to the debate between the structuralists and experimentalists, I address it in Section 4 of this article: (see http://ftp.cs.ucla.edu/pub/stat_ser/r391.pdf)

Your review of Angrist and Paschke book “Mostly Harmless Econometrics” leaves out what in my opinion is the major drawback of their methodology: sole reliance of instrumental variables and failure to express and justify the assumptions that underlie the choice of instruments. Since the choice of instruments rests on the same type of assumptions (ie.,exclusion and exogeneity) that Angrist and Paschke are determined to avoid (for being “unreliable,) readers are left with no discussion of what assumptions do go into the choice of instruments, how they are encoded in a model, what scientific knowledge can be used to defend them, and whether the assumptions have any testable implications.

In your review, you point out that Angrist and Pischke completely avoid the task of model-building; I agree. And I attribute this avoidance, not to lack of good intentions but to lacking mathematical tools necessary for model-building. Angrist and Pischke have deprived themselves of using such tools by making an exclusive commitment to the potential outcome language, while shunning the language of nonparametric structural models. This is something only he/she can appreciate who attempted to solve a problem, from start to end, in both languages, side by side. No philosophy, ideology, or hours of blog discussion can replace the insight one can gain by such an exercise.

2. A discussant named Jack writes:
An economist (econometrician) friend of mine often corresponds with Prof. Pearl, and what I understand is that Pearl believes the econometrics approach to causality is deeply, fundamentally wrong. (And econometricians tend to think Pearl’s approach is fundamentally wrong.) It sounds to me like Pearl was being purposefully snarky.

Judea Pearl replies:
Jack, I think you misunderstood what your friend told you. If you read my papers and books you will come to realize immediately that I believe the econometrics approach to causality is deeply an fundamentally right (I repeat: RIGHT, not WRONG). Though, admittedly, there have been two attempts to distort this approach by influx of researchers from adjacent fields (see my reply to Andrew on this page, or read http://ftp.cs.ucla.edu/pub/stat_ser/r391.pdf

Next, I think you are wrong in concluding that “econometricians tend to think Pearl’s approach is fundamentally wrong”. First, I do not offer anyone “an approach,” I offer mathematical tools to do what researchers say they wish to do, only with less effort and greater clarity; researchers may choose to use or ignore these tools. By analogy, the invention of the microscope was not “an approach” but a new tool.

Second, I do not know a single econometrician who tried my microscope and thought it is “fundamentally wrong”, the dismissals I often hear come invariably from those who refuse to look at the microscope for religious reasons.

Finally, since you went through the trouble of interpreting hearsay and labeling me “purposefully snarky,” I think you owe readers of this blog ONE concrete example where I criticize an economist for reasons that you judge to be unjustified. You be the judge.

3. An Anonymous discussant writes:
Yes, the problem with the econometrics approach is that it lumps together identification, estimation, and probability, so papers look like a Xmas tree. It all starts with chapter 1 in econometrics textbooks and all those assumptions about the disturbance, linearity, etc. Yet most discussions in causality oriented papers revolve around identification and for that you can mostly leave out functional forms, estimation, and probability.

Why carry around reams of parametric notation when it ain’t needed? One wonders how Galileo, Newton, or Franklin ever discovered anything without X’X^(-1)X’Y?

Judea Pearl replies:
To all discussants:
I hear many voices agreeing that statistics education needs a shot of relevancy, and that causality is one area where statistics education has stifled intuition and creativity. I therefore encourage you to submit nominations for the causality in statistics prize, as described in http://www.amstat.org/education/causalityprize/ and http://magazine.amstat.org/blog/2012/11/01/pearl/

Please note that the criteria for the prize do not require fancy formal methods; they are problem-solving oriented. The aim is to build on the natural intuition that students bring with them, and leverage it with elementary mathematical tools so that they can solve simple problems with comfort and confidence (not like their professors). The only skills they need to acquire are: (1) Articulate the question, (2) Specify the assumptions needed to answer it and (3) Determine if the assumptions have testable implications. The reasons we cannot totally dispose of mathematical tools are: (1) scientists have local intuitions about different parts of a problem and only mathematics can put them all together coherently, (2) eventually, these intuitions will need to be combined with data to come up with assessments of strengths and magnitudes (e.g., of effects). We do not know how to combine data with intuition in any other way, except through mathematics.

Recall, Pythagoras theorem served to amplify, not stifle the intuitions of ancient geometers.

I recently attended a seminar presentation by Professor Tom Belin, (AQM RAC seminar, UCLA, November 30, 2012) who spoke on the relationships between the potential outcome model of Neyman, Rubin and Holland, and the structural equation and graphical models which I have been advocating since 1995.

In the last part of the seminar, I made a few comments which led to a lively discussion, as well as clarification ( I hope) of some basic issues which are rarely discussed in the mainstream literature.

Below is a concise summary of my remarks which I present to encourage additional discussion, questions, objections and, of course, new ideas.

Judea Pearl

Summary of my views on the relationships between the potential-outcome (PO) and Structural Causal Models (SCM) frameworks.

Formally, the two frameworks are logically equivalent; a theorem in one is a theorem in the other, and every assumption in one can be translated into an equivalent assumption in the other.

Therefore, the two frameworks can be used interchangeably and symbiotically, as it is done in the advanced literature in the health and social sciences.

However, the PO framework has also spawned an ideological movement that resists this symbiosis and discourages its faithfuls from using SCM or its graphical representation.

This ideological movement (which I call “arrow-phobic”) can be recognized by a total avoidance of causal diagrams or structural equations in research papers, and an exclusive use of “ignorability” type notation for expressing the assumptions that (must) underlie causal inference studies. For example, causal diagrams are meticulously excluded from the writings of Rubin, Holland, Rosenbaum, Angrist, Imbens, and their students who, by and large, are totally unaware of the inferential and representational powers of diagrams.

Formally, this exclusion is harmless because, based on the logical equivalence mentioned above, it is always possible to replace assumptions made in SCM with equivalent, albeit cumbersome assumptions in PO language, and eventually come to the correct conclusions. But practically, the exclusion forces investigators to articulate assumptions whose meaning they do not comprehend, whose plausibility they cannot judge, and whose statistical implications they cannot predict.

The arrow-phobic exclusion can be compared to a prohibition against the use of ‘multiplication’ in arithmetics. Formally, it is harmless, because one can always replace multiplication with addition (e.g., adding a number to itself n times). Yet practically, those who shun multiplication will not get very far in science.

The rejection of graphs and structural models leaves investigators with no process-model guidance and, not surprisingly, it has resulted in a number of blunders which the PO community is not very proud of.

One such blunder is Rosenbaum (2002) and Rubin’s (2007) declaration that “there is no reason to avoid adjustment for a variable describing subjects before treatment”
http://www.cs.ucla.edu/~kaoru/r348.pdf

Another is Hirano and Imbens’ (2001) method of covariate selection, which prefers bias-amplifying variables in the propensity score.
http://ftp.cs.ucla.edu/pub/stat_ser/r356.pdf

The third is the use of ‘principal stratification’ to assess direct and indirect effects in mediation problems. which lead to paradoxical and unintended results.
http://ftp.cs.ucla.edu/pub/stat_ser/r382.pdf

In summary, the PO framework offers a useful analytical tool (i.e.. an algebra of counterfactuals) when used in the context of a symbiotic SCM analysis. It may be harmful however when used as an exclusive and restrictive subculture that discourages the use of process-based tools and insights.

Additional background and technical details on the PO vs. SCM tradeoffs can be found in Section 4 of a tutorial paper (Statistics Surveys)
http://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf
and in a book chapter on the Eight Myths of SEM:
http://ftp.cs.ucla.edu/pub/stat_ser/r393.pdf

Readers might also find it instructive to compare how the two paradigms frame and solve a specific problem from start to end. This comparison is given in Causality (Pearl 2009) pages 81-88, 232-234.