{"id":56,"date":"2009-07-06T04:00:31","date_gmt":"2009-07-06T11:00:31","guid":{"rendered":"http:\/\/www.mii.ucla.edu\/causality\/?p=66"},"modified":"2009-07-06T04:00:31","modified_gmt":"2009-07-06T11:00:31","slug":"resolving-disputes-between-j-pearl-and-d-rubin-on-causal-inference","status":"publish","type":"post","link":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/2009\/07\/06\/resolving-disputes-between-j-pearl-and-d-rubin-on-causal-inference\/","title":{"rendered":"On Myth, Confusion, and Science in Causal Analysis"},"content":{"rendered":"

Andrew Gelman (Columbia) recently wrote a blog post<\/a> motivated by Judea Pearl's paper, "Myth, Confusion, and Science in Causal Analysis.<\/a> " In response, Pearl writes:<\/em><\/strong><\/p>\n

Dear Andrew,<\/p>\n

Thank you for your blog post dated July 5. I appreciate your genuine and respectful quest to explore the differences between the approaches that I and Don Rubin are taking to causal inference.<\/p>\n

In general, I would be the first to rally behind your call for theoretical pluralism (e.g., "It make sense that other theoretical perspectives such as Pearl's could be useful too.") We know that one can prove a theorem in geometry by either geometrical or algebraic methods, depending on the problem and the perspective one prefers to take–only the very dogmatic would label one of the methods "unprincipled".<\/p>\n

My article, "Myth, confusion and Science in Causal Analysis", is written with this dual perspective in mind, fully accommodating the graphical and potential-outcome conceptualizations as interchangeable, "A theorem in one approach is a theorem in another," I wrote.<\/p>\n

However, when adherents of the one-perspective approach make claims that mathematically contradict those derived from the dual-perspective approach, one begins to wonder whether there is something more fundamental at play here.<\/p>\n

In our case, the claims we hear from two adherents of the graph-less one-perspective school is: "there is no reason to avoid adjustment for a variable describing subjects before treatment." And from three adherents of the graph-assisted dual-perspective school we hear: "Adjustment for a variable  describing subjects before treatment may be harmful."<\/p>\n

This is a blatant contradiction that affects every observational study and deserves therefore to be discussed even if we believe in "let one thousand roses bloom."<\/p>\n

One may be tempted to resolve the contradiction by appealing to practical expediencies. For example,<\/p>\n

    \n
  1. Nothing is black and white.<\/li>\n
  2. Perhaps adjustment may be harmful in theory, but is very rare in practice,<\/li>\n
  3. Perhaps the harm is really very small, or<\/li>\n
  4. we do not really know in practice if it is harmful or not, so why worry?<\/li>\n<\/ol>\n

    This line of defense would be agreeable, were it not accompanied with profound philosophical claims that the dual-perspective approach is in some way "unprincipled" and standing (God forbid) "contrary to Bayesianism."<\/p>\n

    The point is that we DO KNOW<\/em> in practice when harm is likely to occur through improper adjustments. The same subjective knowledge that tells us that seat-belt usage does not cause smoking or lung disease also tells us that adjustment for seat-belt usage is likely to introduce bias.<\/p>\n

    Moreover, one can derive this warning in the graph-less notation of potential outcome. So, the question remains: why haven't potential outcome scholars been issuing that warning to their students?<\/p>\n

    The conjecture I made should concern every Bayesian and every educator, for it points beyond M-bias and covariate selection. The conjecture is that the language of "potential outcome" and "ignorability" discourages investigators from articulating and using valuable knowledge which they possess, for example, that seat-belt usage does not cause smoking. Do you know of any study where such a piece of knowledge was used in determining whether treatment assignment is "ignorable" or not? My conjecture is confirmed by potential-outcome practitioners who admit to be using "ignorability" invariably to justify their favorite method of analysis, never as an object to be justified by appeal to causal knowledge.<\/p>\n

    As to indiscriminate conditioning in Bayesian philosophy, the example of controlling for an intermediate variable (between treatment and outcome) should illuminate our discussion. (I do not buy your statement that bias is "tricky to define." It is extremely easy to define, even in Rubin's notation: "Bias" is what you get if you adjust for Z and treatment assignment is not ignorable conditioned on Z. This would suffice for our purposes)<\/p>\n

    You say:<\/p>\n

      \n
    1. A Bayesian analysis can control for intermediate outcomes–that's okay–but then…<\/li>\n
    2. Jennifer and I recommend not controlling for intermediate outcomes.<\/li>\n
    3. You can control for anything, you just then should suitable post process….<\/li>\n
    4. I heard Don Rubin make a similar point… Fisher made this mistake.<\/li>\n<\/ol>\n

      Andrew, I know you did not mean it to sound so indecisive, but it does. Surely, one can always add 17.5 to any number, as long as one remembers to "post-process" and correct the mistake later on. But we are not dealing here with children arithmetic. Why not say it upfront: "You cant arbitrarily add 17,5 to a number and hope you did not do any harm." Even the Mullahs of arithmetic addition would forgive us for saying it that way.<\/p>\n

      If you incorporate an intermediate variable M as a predictor in your propensity score and continue to do matching as if it is just another evidentiary predictor, no post processing will ever help you, except of course, redoing the estimation afresh, with M removed.It will not fix itself by taking more samples. Is Bayesianism so dogmatic as to forbid us from speaking plainly and just say : "Don't condition". (No wonder I once wrote: "Why I am only a half-Bayesian"<\/a>.)<\/p>\n

      True, the great R A Fisher made a similar mistake. But it happened in the context of estimating "direct effects", where one wants to control for the intermediary variable, not in the context of"causal effects," where one wants the intermediaries to vary freely. Incidentally, the repair that Don Rubin offered in the Fisher lecture made things even worse. For example, the direct effect according to Rubin's definition (using principal stratification) is definable only in units absent of indirect effects. This means that a grandfather would be deemed to have no direct effect on his grandson's behavior in families where he has some effect on the father. In linear systems, to take a sharper example, the direct effect would be undefined whenever indirect paths exist from the cause to its effect.<\/p>\n

      Such paradoxical conclusions emanating from a one-perspective culture underscore the wisdom, if not necessity of a dual-perspective analysis, in which the counterfactual notation Yx<\/sub>(u) is governed by the formal semantics of graphs, structural equations and open-mindedness.<\/p>\n

      I just saw Larry Wasserman's comment<\/a>. Larry is right, I do not operate in an "entirely different conceptual framework." I call the [X x Y] –> [0,1] function P(Yx<\/sub> = y) "causal effect" leaving it up to the investigator to form differences P(Y1<\/sub> = y) – P(Y0<\/sub> = y), P(Y8<\/sub> = 3) – P(Y5<\/sub> = 3), or ratios: P(Y1<\/sub> = y) \/ P(Y0<\/sub> = y) or any other comparison that fits fashion and dogma.<\/p>\n

      This does not make for a different conceptual framework, it is the common engineering practice of not wasting precious symbols on trivialities. What does call for a possible realignment of conceptual frameworks is what you tell your students about adjustment for intermediaries, and whether big-brother Bayes approves.<\/p>\n

      Try it.<\/p>\n

      Best,<\/p>\n

      Judea<\/p>\n

      Update 1:<\/font> Please note that a parallel discussion is also underway on Gelman's blog. You may read the
      \ncomments by       clicking here<\/a> .<\/strong><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

      Andrew Gelman (Columbia) recently wrote a blog post motivated by Judea Pearl's paper, "Myth, Confusion, and Science in Causal Analysis. " In response, Pearl writes: Dear Andrew, Thank you for your blog post dated July 5. I appreciate your genuine and respectful quest to explore the differences between the approaches that I and Don Rubin are taking to […]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,29],"tags":[],"class_list":["post-56","post","type-post","status-publish","format-standard","hentry","category-discussion","category-opinion"],"_links":{"self":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/56","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=56"}],"version-history":[{"count":0,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/56\/revisions"}],"wp:attachment":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=56"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=56"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=56"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}