Can do(x) represent practical experiments?
L.B.S., from University of Arizona, questioned whether the do(x) operator can represent realistic actions or experiments: "Even an otherwise perfectly executed randomized experiment may yield perfectly misleading conclusions if, for example, the construct validity of the treatment is zero, e.g., there is a serious confounding. A good example is a study involving injected vitamin E as a treatment for incubated children at risk for retrolental fibroplasia. The randomized experiment indicated efficacy for the injections, but it was soon discovered that the actual effective treatment was opening the pressurized, oxygen-saturated incubators several times per day to give the injections, thus lowering the barometric pressure and oxygen levels in the blood of the infants (Leonard, Major Medical Mistakes). Any statistical analysis would have been misleading in that case."
S.M., from Georgia Institute of Technology, adds:
"Your example of the misleading causal effect, shows the kind of thing that troubles me about the do(x) concept. You do(x) or don't do(x) but something else, and this seems correlated with an effect. But it may be something else that is correlated with do(x) that is the cause and not the do(x) per se."
The do(x) operator was devised to help prevent such troubles. It stands for doing X=x in an ideal experiment, where X, and X alone is manipulated, not any other variable in the model. Mathematics deals with ideal situations, and it is the experimenter's job to make sure that the experimental conditions approximate the mathematical ideal as closely as possible.
In your example of the vitamin E injection (above), there is another variable being manipulated together with X, namely the incubator cover, Z, which turns the experiment into a do(x,z), condition instead of do(x). Thus, the experiment was far from ideal, and far even from the standard experimental protocol, which requires the use of placebo. Had placebo been used (to approximate the requirement of the do(x) operator), the result would not have been biased.
Such sensitivity to deviations from the ideal is not unique to manipulation. Consider the notion of conditional expectation, E(Y|x), which stands for the expected value of Y, given that we observed X=x, and ONLY X=x. If, in practice, we actually observe some other variable, say Z=z, which we fail to report, predictions based of our report would be biased. Yet we hardly find scholars complaining that this sensitivity is a troublesome feature of the conditioning concept. Quite the contrary; scholars find the conditioning operator to be a powerful notational tool for distinguishing among predictions that are based on different sets of observations.
Returning to manipulations, the do(x) operator is a mathematical device that helps us specify explicitly and formally what should be held constant, and what is free to vary in any given experiment. The do-calculus then helps us predict the logical ramifications of such specifications, assuming they are executed faithfully, and assuming we have a valid causal model of the environment.
Best wishes,
========Judea Pearl
Comment by judea — February 21, 2007 @ 11:44 pm