Causal Analysis in Theory and Practice

Readers have written to me that they have encounter criticism from reviewers who question the reliance of
an assumed causal graphs or some of their derivatives. I have encountered this criticism myself and I would like to share my rebuttal with readers, so as to help minimize this line of criticism.

Here is a letter I send to the program committee of a conference in Machine Learning.

Dear Program Committee,
I would like to call your attention to an attitude against which I have been fighting  for the past 30 years and which evidently is still common among researchers and reviewers.

I am referring to remarks made by Reviewer #7 of our paper, which judge model-based, methodological works to be “incremental and unlikely to have much impact”, ostensibly because “we do not have the model” or “one can never tell if the model is correct”, or “where the structures that are reasoned about come from” or “we require knowledge of the process that will be unknown”.

If we examine carefully the developments of ideas in computer science, or even in statistics, we find that most innovative works germinated from model-based questions, assuming that a model is known and asking: (1) what should the world look like for our question to me well-defined, (2) what algorithms can exploit its idiosyncratic features, (3) Can any algorithm produce the desired result and (4) whether the model has any testable implications.

I think conferences and journals should encourage such analytical work to continue and, at the same time, discourage reviewers from dismissing such work as “incremental” because “we do not have the model”.

I hope you share my feelings about this issue. I do not know what we can do to enforce this understanding community wide, but I hope in your communication with reviewers you succeed in conveying this spirit.

Sincerely yours,
Judea Pearl