Causal Analysis in Theory and Practice

A recent discussion on Andrew Gelman’s blog has touched on some interesting points concerning the teaching of causality in econometric and statistics classes (link here). I responded to some of the discussants and, below, I share my replies with readers of this blog.

1. Andrew Gelman asked why the review in http://ftp.cs.ucla.edu/pub/stat_ser/r395.pdf is critical of econometrics, “I thought that causality was central to econometrics; see, for example, Angrist and Pischke’s book .”

Judea Pearl replies:
Causality is indeed central to econometrics. Our survey of econometric textbooks http://ftp.cs.ucla.edu/pub/stat_ser/r395.pdf is critical of econometric education today, not of econometric methodology proper. Econometric models, from the time of Haavelmo (1943), have been and remained causal (see http://ftp.cs.ucla.edu/pub/stat_ser/r391.pdf ) despite two attempted hijacking, first by regressionists, and second by “quasi-experimentalists,” like Angrist and Paschke (AP). The six textbooks we reviewed reflect a painful recovery from the regressionist assault which more or less disappeared from serious econometric research, but is still obfuscating authors of econometric textbooks.

As to the debate between the structuralists and experimentalists, I address it in Section 4 of this article: (see http://ftp.cs.ucla.edu/pub/stat_ser/r391.pdf)

Your review of Angrist and Paschke book “Mostly Harmless Econometrics” leaves out what in my opinion is the major drawback of their methodology: sole reliance of instrumental variables and failure to express and justify the assumptions that underlie the choice of instruments. Since the choice of instruments rests on the same type of assumptions (ie.,exclusion and exogeneity) that Angrist and Paschke are determined to avoid (for being “unreliable,) readers are left with no discussion of what assumptions do go into the choice of instruments, how they are encoded in a model, what scientific knowledge can be used to defend them, and whether the assumptions have any testable implications.

In your review, you point out that Angrist and Pischke completely avoid the task of model-building; I agree. And I attribute this avoidance, not to lack of good intentions but to lacking mathematical tools necessary for model-building. Angrist and Pischke have deprived themselves of using such tools by making an exclusive commitment to the potential outcome language, while shunning the language of nonparametric structural models. This is something only he/she can appreciate who attempted to solve a problem, from start to end, in both languages, side by side. No philosophy, ideology, or hours of blog discussion can replace the insight one can gain by such an exercise.

2. A discussant named Jack writes:
An economist (econometrician) friend of mine often corresponds with Prof. Pearl, and what I understand is that Pearl believes the econometrics approach to causality is deeply, fundamentally wrong. (And econometricians tend to think Pearl’s approach is fundamentally wrong.) It sounds to me like Pearl was being purposefully snarky.

Judea Pearl replies:
Jack, I think you misunderstood what your friend told you. If you read my papers and books you will come to realize immediately that I believe the econometrics approach to causality is deeply an fundamentally right (I repeat: RIGHT, not WRONG). Though, admittedly, there have been two attempts to distort this approach by influx of researchers from adjacent fields (see my reply to Andrew on this page, or read http://ftp.cs.ucla.edu/pub/stat_ser/r391.pdf

Next, I think you are wrong in concluding that “econometricians tend to think Pearl’s approach is fundamentally wrong”. First, I do not offer anyone “an approach,” I offer mathematical tools to do what researchers say they wish to do, only with less effort and greater clarity; researchers may choose to use or ignore these tools. By analogy, the invention of the microscope was not “an approach” but a new tool.

Second, I do not know a single econometrician who tried my microscope and thought it is “fundamentally wrong”, the dismissals I often hear come invariably from those who refuse to look at the microscope for religious reasons.

Finally, since you went through the trouble of interpreting hearsay and labeling me “purposefully snarky,” I think you owe readers of this blog ONE concrete example where I criticize an economist for reasons that you judge to be unjustified. You be the judge.

3. An Anonymous discussant writes:
Yes, the problem with the econometrics approach is that it lumps together identification, estimation, and probability, so papers look like a Xmas tree. It all starts with chapter 1 in econometrics textbooks and all those assumptions about the disturbance, linearity, etc. Yet most discussions in causality oriented papers revolve around identification and for that you can mostly leave out functional forms, estimation, and probability.

Why carry around reams of parametric notation when it ain’t needed? One wonders how Galileo, Newton, or Franklin ever discovered anything without X’X^(-1)X’Y?

Judea Pearl replies:
To all discussants:
I hear many voices agreeing that statistics education needs a shot of relevancy, and that causality is one area where statistics education has stifled intuition and creativity. I therefore encourage you to submit nominations for the causality in statistics prize, as described in http://www.amstat.org/education/causalityprize/ and http://magazine.amstat.org/blog/2012/11/01/pearl/

Please note that the criteria for the prize do not require fancy formal methods; they are problem-solving oriented. The aim is to build on the natural intuition that students bring with them, and leverage it with elementary mathematical tools so that they can solve simple problems with comfort and confidence (not like their professors). The only skills they need to acquire are: (1) Articulate the question, (2) Specify the assumptions needed to answer it and (3) Determine if the assumptions have testable implications. The reasons we cannot totally dispose of mathematical tools are: (1) scientists have local intuitions about different parts of a problem and only mathematics can put them all together coherently, (2) eventually, these intuitions will need to be combined with data to come up with assessments of strengths and magnitudes (e.g., of effects). We do not know how to combine data with intuition in any other way, except through mathematics.

Recall, Pythagoras theorem served to amplify, not stifle the intuitions of ancient geometers.