Causal Analysis in Theory and Practice

Judea Pearl Writes:

Last week I attended a causal inference symposium at the University of Michigan, and had a very lively discussion with James Heckman (Chicago, economics) on causal reasoning in econometrics, statistics and computer science. Video and slides of the two lectures can be watched here: http://www.psc.isr.umich.edu/pubs/video-tapes.html

In the QA session (not in the video), I described the problems of transportability and external validity, and their solutions according to:
http://ftp.cs.ucla.edu/pub/stat_ser/r372.pdf
http://ftp.cs.ucla.edu/pub/stat_ser/r390.pdf

Heckman asked: What makes this problem different from the one that economists solve routinely — when they find a new distribution that differs from the one they estimated, they simply re-estimate the parameters by which the two differ and keep those on which they agree.

My answer stressed three facts that should be kept in mind when dealing with “transporatability”:
1. We cannot speak here about differing “distributions” because transportability is a causal, not statistical problem. In other words, what needs to be re-estimated depends not on the two “distributions” but on the causal story behind the distributions. (This is shown vividly in Example 2 of R-372).

2. We are now dealing with the task of transporting “experimental findings” (e.g., causal effects), not distributions, from a place where they are available to a place where they are not estimable.

3. We cannot even speak about re-estimating “parameters” because the problem is entirely non-parametric.

More comments on audience questions will follow.

We received the following query from Megan Murphy (ASA):
Dr. Pearl,
I received the following question regarding the Causality in Statistics Education prize on twitter. I’m not sure how to answer this, perhaps you can help?

Would entries using Neyman-Rubin model even be considered? RT @AmstatNews: Causality in Statistics Education #prize magazine.amstat.org/blog/2012/11/0…

Judea Answers:
“Of course! The criteria for evaluation specifically state: ‘in some mathematical language (e.g., counterfactuals, equations, or graphs)’ giving no preference to any of the three notational systems. The criteria stress capabilities to perform specific inference tasks, regardless of the tools used in performing the tasks.

For completeness, I re-list below the evaluation criteria:

• The extent to which the material submitted equips students with skills needed for effective causal reasoning. These include:

—1a. Ability to correctly classify problems, assumptions, and claims into two distinct categories: causal vs. associational

—1b. Ability to take a given causal problem and articulate in some mathematical language (e.g., counterfactuals, equations, or graphs) both the target quantity to be estimated and the assumptions one is prepared to make (and defend) to facilitate a solution

—1c. Ability to determine, in simple cases, whether control for covariates is needed for estimating the target quantity, what covariates need be controlled, what the resulting estimand is, and how it can be estimated using the observed data

—1d. Ability to take a simple scenario (or model), determine whether it has statistically testable implications, and apply data to test the assumed scenario

• The extent to which the submitted material assists statistics instructors in gaining an understanding of the basics of causal inference (as outlined in 1a-d) and prepares them to teach these basics in undergraduate and lower-division graduate classes in statistics.

Those versed in the Neyman-Rubin model are most welcome to submit nominations.

Note, however, that nominations will be evaluated on ALL four skills, 1a – 1d.
Judea

I recently attended a seminar presentation by Professor Tom Belin, (AQM RAC seminar, UCLA, November 30, 2012) who spoke on the relationships between the potential outcome model of Neyman, Rubin and Holland, and the structural equation and graphical models which I have been advocating since 1995.

In the last part of the seminar, I made a few comments which led to a lively discussion, as well as clarification ( I hope) of some basic issues which are rarely discussed in the mainstream literature.

Below is a concise summary of my remarks which I present to encourage additional discussion, questions, objections and, of course, new ideas.

Judea Pearl

Summary of my views on the relationships between the potential-outcome (PO) and Structural Causal Models (SCM) frameworks.

Formally, the two frameworks are logically equivalent; a theorem in one is a theorem in the other, and every assumption in one can be translated into an equivalent assumption in the other.

Therefore, the two frameworks can be used interchangeably and symbiotically, as it is done in the advanced literature in the health and social sciences.

However, the PO framework has also spawned an ideological movement that resists this symbiosis and discourages its faithfuls from using SCM or its graphical representation.

This ideological movement (which I call “arrow-phobic”) can be recognized by a total avoidance of causal diagrams or structural equations in research papers, and an exclusive use of “ignorability” type notation for expressing the assumptions that (must) underlie causal inference studies. For example, causal diagrams are meticulously excluded from the writings of Rubin, Holland, Rosenbaum, Angrist, Imbens, and their students who, by and large, are totally unaware of the inferential and representational powers of diagrams.

Formally, this exclusion is harmless because, based on the logical equivalence mentioned above, it is always possible to replace assumptions made in SCM with equivalent, albeit cumbersome assumptions in PO language, and eventually come to the correct conclusions. But practically, the exclusion forces investigators to articulate assumptions whose meaning they do not comprehend, whose plausibility they cannot judge, and whose statistical implications they cannot predict.

The arrow-phobic exclusion can be compared to a prohibition against the use of ‘multiplication’ in arithmetics. Formally, it is harmless, because one can always replace multiplication with addition (e.g., adding a number to itself n times). Yet practically, those who shun multiplication will not get very far in science.

The rejection of graphs and structural models leaves investigators with no process-model guidance and, not surprisingly, it has resulted in a number of blunders which the PO community is not very proud of.

One such blunder is Rosenbaum (2002) and Rubin’s (2007) declaration that “there is no reason to avoid adjustment for a variable describing subjects before treatment”
http://www.cs.ucla.edu/~kaoru/r348.pdf

Another is Hirano and Imbens’ (2001) method of covariate selection, which prefers bias-amplifying variables in the propensity score.
http://ftp.cs.ucla.edu/pub/stat_ser/r356.pdf

The third is the use of ‘principal stratification’ to assess direct and indirect effects in mediation problems. which lead to paradoxical and unintended results.
http://ftp.cs.ucla.edu/pub/stat_ser/r382.pdf

In summary, the PO framework offers a useful analytical tool (i.e.. an algebra of counterfactuals) when used in the context of a symbiotic SCM analysis. It may be harmful however when used as an exclusive and restrictive subculture that discourages the use of process-based tools and insights.

Additional background and technical details on the PO vs. SCM tradeoffs can be found in Section 4 of a tutorial paper (Statistics Surveys)
http://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf
and in a book chapter on the Eight Myths of SEM:
http://ftp.cs.ucla.edu/pub/stat_ser/r393.pdf

Readers might also find it instructive to compare how the two paradigms frame and solve a specific problem from start to end. This comparison is given in Causality (Pearl 2009) pages 81-88, 232-234.

Eric Neufeld (University of Saskatchewan/Canada) asks:

I have been interested in giving a lecture to lay people about your work, but would like to compare/contrast it with Rubin’s work. I understand how strongly you disagree! But I would appreciate it if you could point me to a couple of your articles that lay the argument out in an accessible way.

Judea Pearl replies:

I have written a fairly decent section on Rubin’s model in Statistics Surveys:
http://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf
and a less technical one in:
http://ftp.cs.ucla.edu/pub/stat_ser/r348-warning.pdf
(unpublished but highly recommended.)

Important to stress: I do not disagree with Rubin’s work (it is subsumed by structural modeling), I am merely mused by the tunnel-visioned culture that this work has engendered.

The American Statistical Association has announced a new Prize,
“Causality in Statistics Education”, aimed to encourage the teaching of
basic causal inference in introductory statistics courses.

The motivation for the prize is discussed in an interview I gave to Ron Wasserstein:
http://magazine.amstat.org/blog/2012/11/01/pearl/

Nomination procedures and selection criteria can be found here:
http://www.amstat.org/education/causalityprize/

I hope readers of this blog will participate, either by innovating new
ways of teaching causation or by identifying candidates deserving of the prize.
Judea

The slides used in a recent UAI tutorial on
“Graphical Models for Causal Inference” are now
available for public view and public use.
click on

http://ftp.cs.ucla.edu/pub/stat_ser/uai12-mohan-pearl.pdf

The slides were prepared by Karthika Mohan
and the topics include:

1. probabilitic graphical models
2. Markov compatibility
3. d-separation
4. Interventions
5. Causal effects identification
6. do-Calculus
7. C-components
8. Counterfactuals
9. Markov Equivalence
10. MAGs
11. Confounding Equivalence
12. Instrumental Variables
13. Verma’s constraints

Enjoy

The European Journal of Personality just published an article by James Lee, titled
“Correlation and Causation in the Study of Personality”
European Journal of Personality, Eur.J.Pers. 26: 372-390 (2012) DOI:10.1002/per.1863.
Link: http://onlinelibrary.wiley.com/doi/10.1002/per.1863/pdf,
or here.

Lee’s article is followed by Open Peer Commentaries
http://onlinelibrary.wiley.com/doi/10.1002/per.1865/full,
or here.

(Strikingly, the commentary by Rolf Steyer declares the do-operator to be self-contradictory. I trust readers of this blog to spot Steyer’s error right away. If not, I will post.)

Another recent paper on causation in psychological research is the one by Shadish and Sullivan,
“Theories of Causation in Psychological Science”
In Harris Cooper (Ed-in-Chief), APA Handbook of Research Methods in Psychology, Volume 1, pp. 23-52, 2012.
http://www.cs.ucla.edu/~kaoru/shadish-sullivan12.pdf

While these papers indicate a healthy awakening of psychological researchers to recent advances in causal inference, the field is still dominated by authors who have not heard about model-based covariate selection, testable implications, nonparametric identification, bias amplification, mediation formulas and more.

Much to do, much to discuss,
Judea

Andrew Gelman and his blog readers followed-up with the previous discussion (link here) on his methods to address issues about causal inference and transportability of causal effects based on his “hierarchical modeling” framework, and I just posted my answer.

This is the general link for the discussion:
http://andrewgelman.com/2012/07/examples-of-the-use-of-hierarchical-modeling-to-generalize-to-new-settings/

Here is my answer:
http://andrewgelman.com/2012/07/examples-of-the-use-of-hierarchical-modeling-to-generalize-to-new-settings/#comment-92499

Cheers,
Bareinboim

In the past week, I have been engaged in a discussion with Andrew Gelman and his blog readers regarding causal inference, selection bias, confounding, and generalizability. I was trying to understand how his method which he calls “hierarchical modelling” would handle these issues and what guarantees it provides. Unfortunately, I could not reach an understanding of Gelman’s method (probably because no examples were provided).

Still, I think that this discussion having touched core issues of scientific methodology would be of interest to readers of this blog, the link follows:
http://andrewgelman.com/2012/07/long-discussion-about-causal-inference-and-the-use-of-hierarchical-models-to-bridge-between-different-inferential-settings/

Previous discussions took place regarding Rubin and Pearl’s dispute, here are some interesting links:
http://andrewgelman.com/2009/07/disputes_about/
http://andrewgelman.com/2009/07/more_on_pearlru/
http://andrewgelman.com/2009/07/pearls_and_gelm/
http://andrewgelman.com/2012/01/judea-pearl-on-why-he-is-only-a-half-bayesian/

If anyone understands how “hierarchical modeling” can solve a simple toy problem (e.g., M-bias, control of confounding, mediation, generalizability), please share with us.

Cheers,
Bareinboim

Dear friends in causality research,

This communication highlights a few meetings in the summer of 2012, that should be of interest to causality researchers. Naturally, these are biased in favor of those that were brought to my attention. If you know of more such meetings, feel free to post.

1.
July 22-26, 2012, Toronto, Canada
Annual Conference of the Association for Advancement of
Artificial Intelligence, AAAI-12,
http://www.aaai.org/Conferences/AAAI/2012/

I will present a lecture on “The Mechanization of Causal Inference, a Mini Turing-Test and Beyond”`
http://www.aaai.org/Conferences/AAAI/2012/aaai12turing.php
and Elias Bareinboim will present a completeness result for the transportability problem.
http://ftp.cs.ucla.edu/pub/stat_ser/r390.pdf

2.
Joint Statistical Meeting
JSM 2012, San Diego, CA, July 28-Aug 3, 2012
There are 73 papers and meetings on causal inference listed in the program, here they are: http://www.amstat.org/meetings/jsm/2012/onlineprogram/KeywordSearchResults.cfm

These include a day-long course on Targeted Learning: Causal Inference for Observational and Experimental Data
CE_08C Sun, 7/29/2012, 8:30 AM – 5:00 PM HQ-Indigo E
by Maya Petersen, Sherri Rose, Mark van der Laan, http://www.amstat.org/meetings/jsm/2012/onlineprogram/AbstractDetails.cfm?abstractid=304318

And J. Pearl tutorial “Causal Inference in Statistics: A gentle introduction” Sunday, July 17 4-6pm http://www.amstat.org/meetings/jsm/2012/onlineprogram/AbstractDetails.cfm?abstractid=304318

—————An Interesting Observation ——
In 2002, JSM-2002 had only 13 papers on causal inference, By any gauge, 73 is a positive sign of progress in the field.
—————end of interesting observation ——

3.
UAI2012, Catalina, CA, August 15-17, 2012
(Uncertainty in Artificial Intelligence)

Workshop on Causal Structure Learning, Aug. 18
http://www.stat.washington.edu/tsr/uai-causal-structure-learning-workshop/

Alison Gopnik will speak on “Babies, Brain and Bayes (Banquet Speech), Thursday, Aug. 16.,
and I will speak on “Do-Calculus Revisited” Aug. 17, 1:30 pm
http://www.auai.org/uai2012/invited.shtml

4.
Workshop on Networks Processes and Causality
Sept 3-6, 2012, Menorca, Spain
http://people.tuebingen.mpg.de/networks-workshop/

5.
MLSP 2012 Special Session on Causal Discovery
IEEE Workshop on Machine Learning for Signal Processing (MLSP 2012)
September 23-26 2012, Santander, Spain
http://mlsp2012.conwiz.dk/index.php?id=3D62

6.
Symposium on Causal Inference, University of Michigan,
December 12, 2012. Contact: Professor Yu Xie.

Miscellaneous
7.
Larry Wasserman has a new blog, and dedicated a page to befriending “causality”.
http://normaldeviate.wordpress.com/2012/06/18/48/

8.
A new book “Causality: Statistical Perspectives and Applications”,
C. Berzuini, P. Dawid and L. Bernardinelli (Eds.)
has just been published by Wiley (Chihester) July 2012: see
http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470665564.html .

Perpetual online access to the book is available on:
http://onlinelibrary.wiley.com/doi/10.1002/9781119945710.fmatter/summary

9.
Another new book appeared this month
R.H. Hoyle (Ed.) Handbook of Structural Equation Modeling, New York: Guilford Press.
It contains my chapter on “The Causal Foundations of SEM”
ttp://ftp.cs.ucla.edu/pub/stat_ser/r370.pdf

10.
Finally, the UCLA fruit basket offers a few fresh items, see http://bayes.cs.ucla.edu/csl_papers.html
One item of interest to economists-educators is:
Chen and Pearl “A Critical Examination of Econometrics Textbooks,” where we survey six influential econometric textbooks in terms of their mathematical treatment of causal concepts. The conclusions are revealing, if you click on http://ftp.cs.ucla.edu/pub/stat_ser/r395.pdf (And if you have ideas on reforming econometric education, please share.)

Wishing you an insightful and productive summer
Best,
Judea Pearl