Contributor: Judea Pearl<\/p>\n
If you search Google for \u201cSimpson\u2019s paradox,\u201d as I\u00a0did yesterday, you will get 111,000 results, more than any other\u00a0statistical paradox that I could name. What elevates\u00a0this innocent\u00a0reversal of association to \u201cparadoxical\u201d status, and why it has\u00a0captured the fascination of statisticians, mathematicians\u00a0and philosophers for over a century are questions\u00a0that we discussed at length on this (and other) blogs. The reason\u00a0I am back to this topic is the publication of four recent\u00a0papers that give us a panoramic view\u00a0at how the understanding\u00a0of causal reasoning has progressed in communities that do not\u00a0usually participate in our discussions.<\/p>\n
As readers of this blog recall,\u00a0I have been trying since the publication of\u00a0Causality\u00a0<\/em>(2000) to convince statisticians, philosophers\u00a0and other scientific communities that Simpson\u2019s paradox is: (1) a product of wrongly applied causal principles, and (2) that it can be fully resolved using modern tools of causal inference.<\/p>\n The four papers to be discussed do not fully agree with the proposed resolution.<\/p>\n To reiterate my position, Simpson\u2019s paradox\u00a0is (quoting Lord Russell) \u201canother relic of a bygone age,\u201d an\u00a0age when we believed that every\u00a0peculiarity in the data can be understood and resolved by statistical\u00a0means. Ironically, Simpson’s\u00a0paradox has actually become\u00a0an educational tool for\u00a0demonstrating the limits of statistical methods, and why causal,\u00a0rather than statistical considerations are necessary\u00a0to avoid paradoxical interpretations of data. For example,\u00a0our recent book Causal Inference in Statistics: A Primer<\/em>, uses Simpson\u2019s paradox at the very beginning (Section 1.1),\u00a0to show\u00a0students the inevitability of causal thinking\u00a0and the futility of trying to interpret data using statistical\u00a0tools alone. See\u00a0http:\/\/bayes.cs.ucla.edu\/ Thus, my interest in the four recent articles\u00a0stems primarily from curiosity to gauge the penetration of\u00a0causal ideas into communities that were not\u00a0intimately involved in the development of graphical\u00a0or counterfactual models.\u00a0Discussions of Simpson\u2019s paradox provide a sensitive litmus\u00a0test to measure\u00a0the acceptance of modern causal thinking.\u00a0\u201cTalk to me about Simpson,\u201d I often say to\u00a0friendly colleagues,\u00a0\u201cand I will tell you how far you are on the causal trail.\u201d\u00a0(Unfriendly colleagues balk at the idea that there is\u00a0a trail they might have missed.)<\/p>\n The four papers for discussion are the following:<\/p>\n 1. 2. 3. 4. \u2014\u2014\u2014\u2014\u2014\u2014- Discussion \u2014\u2014\u2014\u2014\u2014\u2014-<\/p>\n 1.\u00a0Molina and Bigelow 2016 (MB)<\/p>\n I will start the discussion with\u00a0Molina and Bigelow 2016 (MB) because the\u00a0Stanford Encyclopedia of Philosophy enjoys\u00a0both high visibility and an aura of authority.\u00a0MB\u2019s new entry is a welcome revision of\u00a0their previous article (2004) on \u201cSimpson\u2019s Paradox,\u201d which was written almost entirely from the perspective of\u00a0\u201cprobabilistic causality,\u201d echoing Reichenbach, Suppes,\u00a0Cartwright, Good, Hesslow, Eells, to cite a few.<\/p>\n Whereas the previous version characterizes Simpson\u2019s\u00a0reversal as \u201cA Logically Benign, empirically Treacherous\u00a0Hydra,\u201d the new version dwarfs the dangers of that Hydra\u00a0and correctly states that Simpson\u2019s paradox poses\u00a0problem only for \u201cphilosophical programs that aim\u00a0to eliminate or reduce causation to\u00a0regularities and relations between probabilities.\u201d\u00a0Now, since the \u201cprobabilistic causality\u201d program\u00a0is fairly much abandoned in the past two decades,\u00a0we can safely conclude that Simpson\u2019s reversal poses\u00a0no problem to us mortals. This is reassuring.<\/p>\n MB also acknowledge the role that graphical tools\u00a0play in deciding whether one should base a decision on\u00a0the aggregate population or on the partitioned\u00a0subpopulations, and in testing one\u2019s hypothesized model.<\/p>\n My only disagreement with the MB’s article is that it does not\u00a0go all the way towards divorcing the discussion from the\u00a0molds, notation and examples of the \u201cprobabilistic\u00a0causation\u201d era and, naturally, proclaim the\u00a0paradox \u201cresolved.\u201d By shunning modern notation\u00a0like do(x), Yx<\/sub>, or their equivalent,\u00a0the article gives the impression\u00a0that Bayesian conditionalization, as in P(y|x), is still adequate for discussing Simpson\u2019s paradox, its ramifications\u00a0and its resolution. It is not.<\/p>\n In particular, this notational orthodoxy makes the discussion\u00a0of the Sure Thing Principle (STP) incomprehensible\u00a0and obscures the reason why Simpson\u2019s\u00a0reversal does not constitute a counter example to STP. Specifically, it does not tell readers that causal independence\u00a0is a necessary condition for the validity of the STP,\u00a0(i.e., actions should not change the size of the subpopulations)\u00a0and this independence is violated in the counterexample\u00a0that Blyth contrived in 1972.\u00a0(See\u00a0http:\/\/ftp.cs.ucla. I will end with a humble recommendation to the editors of the\u00a0Stanford Encyclopedia of Philosophy<\/em>. Articles concerning\u00a0causation should be written in a language that permits\u00a0authors to distinguish causal from statistical dependence.\u00a0I am sure future authors in this series would enjoy the\u00a0freedom of saying \u201ctreatment does not change gender,\u201d\u00a0something they cannot say today, using Bayesian conditionalization. However, they will not do so on their own, unless you\u00a0tell them (and their reviewers) explicitly that it is\u00a0ok nowadays to deviate from the language of Reichenbach\u00a0and Suppes and formally state:\u00a0P(gender|do(treatment)) = P(gender).<\/p>\n Editorial guidance can play an incalculable role in the progress\u00a0of science.<\/p>\n 2. Comments on Spanos (2016)<\/p>\n In 1988, the British econometrician John Denis Sargan\u00a0gave the following definition of an \u201ceconomic model\u201d:\u00a0\u201cA model is the specification of the probability distribution\u00a0for a set of observations. A structure is the specification\u00a0of the parameters of that distribution.\u201d\u00a0(Lectures on Advanced Econometric Theory<\/em>\u00a0(1988, p.27))<\/p>\n This definition, still cited in advanced econometric books (e.g., Cameron and Trivdi (2009)\u00a0Microeconometrics<\/em>)\u00a0has served as a credo to a school of economics\u00a0that has never elevated itself from the data-first paradigm\u00a0of statistical thinking. Other prominent leaders of\u00a0this school include Sir David Hendry, who wrote:\u00a0\u201cThe joint density is the basis: SEMs (Structural Equation\u00a0Models) are merely an interpretation of that.\u201d\u00a0Members of this school are unable to internalize the hard\u00a0fact that statistics, however refined, cannot provide the\u00a0information that economic models must encode to be of use\u00a0to policy making. For them, a model is just a compact\u00a0encoding of the density function underlying the data,\u00a0so, two models encoding the same density function are\u00a0deemed interchangeable.<\/p>\n Spanos article is a vivid example of how this\u00a0statistics-minded culture copes with causal problems.\u00a0Naturally, Spanos attributes the peculiarities of\u00a0Simpson\u2019s reversal to what he calls \u201cstatistical\u00a0misspecification, Spanos\u2019 conditions for \u201cstatistical adequacy\u201d are formulated\u00a0in the context of the Linear Regression Model and invoke\u00a0strictly statistical notions such as normality,\u00a0linearity, independence etc. None of them applies\u00a0to the binary case of {treatment, gender, outcome}\u00a0in which Simpson\u2019s paradox is usually cast.\u00a0I therefore fail to see why replacing \u201cgender\u201d with\u00a0\u201cblood pressure\u201d would turn an association from \u201cspurious\u201d to\u00a0\u201ctrustworthy\u201d.<\/p>\n Perhaps one of our readers can illuminate the rest\u00a0of us how to interpret this new proposal.\u00a0I am at a total loss.<\/p>\n For fairness, I should add that most economists\u00a0that I know\u00a0have second thoughts about Sargan\u2019s definition, and\u00a0claim to understand\u00a0the distinction between\u00a0structural and statistical models. This distinction,\u00a0unfortunately, is still badly missing from econometric textbooks,\u00a0see\u00a0http:\/\/ftp.cs. 3. Memetea (2015)<\/p>\n Among the four papers under consideration, the one\u00a0by Memetea, is by far the most advanced, comprehensive and\u00a0forward thinking. As a thesis written in a philosophy\u00a0department, Memetea treatise is unique in that it makes a\u00a0serious and successful effort to break away from\u00a0the cocoon of\u00a0\u201cprobabilistic causality\u201d and examines Simpson\u2019s paradox\u00a0to the light of modern causal inference,\u00a0including graphical models, do-calculus, and\u00a0counterfactual theories.<\/p>\n Memetea agrees with our view that\u00a0the paradox is causal in nature, and that\u00a0the tools of modern causal analysis are essential\u00a0for its resolution. She disagrees however with\u00a0my provocative claim that the paradox is \u201cfully resolved\u201d.\u00a0The areas where she finds the resolution wanting\u00a0are mediation cases in which the direct effect (DE)\u00a0differs in sign from the total effect (TE).\u00a0The classical example of such cases (Hesslow 1976)\u00a0tells of a birth control pill that is suspected of producing\u00a0thrombosis in women and, at the same time, has a negative\u00a0indirect effect on thrombosis by reducing the rate of\u00a0pregnancies (pregnancy is known to encourage thrombosis).<\/p>\n I have always argued that Hesslow\u2019s example has nothing to\u00a0do with Simpson\u2019s paradox because it compares apples and\u00a0oranges, namely, it compare direct vs. total effects\u00a0where reversals are commonplace. In other words,\u00a0Simpson\u2019s reversal evokes no surprise in such cases. For example, I wrote, \u201cwe are not at all surprised when\u00a0smallpox inoculation carries risks of fatal reaction,\u00a0yet reduces overall mortality by irradicating smallpox.\u00a0The direct effect (fatal reaction) in this case is\u00a0negative for every subpopulation, yet\u00a0the total effect (on mortality) is positive for the\u00a0population as a whole.\u201d (Quoted from\u00a0http:\/\/ftp.cs.ucla.edu\/ Memetea is not satisfied with this answer.\u00a0Her condition for resolving Simpson\u2019s paradox\u00a0requires that the analyst be told whether it is\u00a0the direct or the total effect that should be the target\u00a0of investigation. This would require, of course, that\u00a0the model includes information about the investigator\u2019s\u00a0ultimate aims, whether alternative interventions are\u00a0available (e.g. to prevent pregnancy), whether the\u00a0study result will be used by a policy maker or\u00a0a curious scientist, whether legal restrictions\u00a0(e.g., on sex discrimination) apply to the direct or\u00a0the total effect, and so on.\u00a0In short, the entire spectrum of scientific and social knowledge\u00a0should enter into the causal model before we can determine,\u00a0in any given scenario, whether it is the direct or indirect\u00a0effect that warrants our attention.<\/p>\n This is a rather tall order to satisfy given that\u00a0our investigators are fairly good in determining\u00a0what their research problem is.\u00a0It should perhaps serve as a realizable goal for\u00a0artificial intelligence researchers among us, who aim to build an\u00a0automated scientist some day, capable of reasoning\u00a0like our best investigators.\u00a0I do not believe though that we need to wait for that\u00a0day to declare Simpson\u2019s paradox \u201cresolved\u201d. Alternatively,\u00a0we can declare it resolved modulo the ability of\u00a0investigators to define their research problems.<\/p>\n 4.\u00a0Comments on Bandyopadhyay, etal (2015)<\/p>\n There are several motivations behind the resistance\u00a0to characterize Simpson\u2019s paradox as a causal phenomenon.\u00a0Some resist because causal relationships are not\u00a0part of their scientific vocabulary, and some because\u00a0they think they have discovered a more cogent explanation,\u00a0which is perhaps easier to demonstrate or communicate.<\/p>\n Spanos\u2019s article represents the first group, while\u00a0Bandyopadhyay etal\u2019s represents the second.\u00a0They simulated Simpson\u2019s reversal using urns and balls and argued that, since there\u00a0are no interventions involved in this setting, merely\u00a0judgment of conditional probabilities, the fact that people tend to make wrong judgments in this setting\u00a0proves that Simpson\u2019s surprise is rooted in arithmetic illusion,\u00a0not in causal misinterpretation.<\/p>\n I have countered this argument in\u00a0http:\/\/ftp.cs.ucla.edu\/pub\/ \u201cIn explaining the surprise, we must first distinguish between \u2018Simpson\u2019s reversal\u2019 and \u2018Simpson\u2019s paradox\u2019; the former being an arithmetic phenomenon\u00a0in the calculus of proportions, the latter a psychological\u00a0phenomenon that evokes surprise and disbelief.\u00a0A full understanding of Simpson\u2019s paradox\u00a0should explain why an innocent arithmetic reversal of\u00a0an association, albeit uncommon,\u00a0came to be regarded as `paradoxical,\u2019 and why it has\u00a0captured the fascination of\u00a0statisticians, mathematicians and philosophers for over a century\u00a0(though it was first labeled \u2018paradox\u2019 by Blyth (1972)) .<\/p>\n \u201cThe arithmetics of proportions has its share of\u00a0peculiarities, no doubt, but these tend to\u00a0become objects of curiosity once they have been demonstrated\u00a0and explained away by examples.\u00a0For instance, naive students of probability may expect\u00a0the average of a product to equal the product\u00a0of the averages but quickly learn to guard against such\u00a0expectations, given a few counterexamples. Likewise,\u00a0students expect an association measured\u00a0in a mixture distribution to equal a weighted average\u00a0of the individual associations. They are surprised, therefore,\u00a0when ratios of sums, (a+b)\/(c+d), are found to be ordered differently\u00a0than individual ratios, a\/c\u00a0and b\/d.1<\/sup>\u00a0Again, such arithmetic peculiarities are quickly\u00a0accommodated by seasoned students as reminders\u00a0against simplistic reasoning.<\/p>\n \u201cIn contrast, an arithmetic peculiarity becomes \u2018paradoxical\u2019 when it clashes with\u00a0deeply held convictions that the peculiarity is impossible,\u00a0and this occurs when one takes seriously the\u00a0causal implications of Simpson\u2019s reversal in decision-making contexts.\u00a0 Reversals are indeed impossible\u00a0whenever the third variable, say age or gender, stands for a\u00a0pre-treatment covariate because, so the reasoning goes,\u00a0no drug can be harmful to both males and\u00a0females yet beneficial to the population as a whole.\u00a0The universality of this intuition reflects\u00a0a deeply held and valid conviction that such a drug\u00a0is physically impossible.\u00a0 Remarkably, such impossibility can be derived\u00a0mathematically in the calculus of causation in the form of\u00a0a \u2018sure-thing\u2019 theorem (Pearl, 2009, p. 181):<\/p>\n \u2018An action\u00a0A<\/em>\u00a0that increases the probability of an event\u00a0B\u00a0<\/em>in each subpopulation (of\u00a0C<\/em>) must also increase the probability of\u00a0B<\/em>\u00a0in the population as a whole, provided that the action\u00a0does not change the distribution of the subpopulations.\u20192<\/sup><\/p>\n \u201cThus, regardless of whether effect size is measured by\u00a0the odds ratio or other comparisons, regardless of whether\u00a0Z\u00a0<\/em>\u00a0is a confounder or not, and regardless of whether we have\u00a0the correct causal structure on hand, our intuition should\u00a0be offended by any effect reversal that appears to accompany the aggregation of data.<\/p>\n \u201cI am not aware of another condition that rules out effect\u00a0reversal with comparable assertiveness and generality,\u00a0requiring only that\u00a0Z<\/em>\u00a0not be affected by our action,\u00a0a requirement satisfied by all treatment-independent covariates\u00a0Z.\u00a0<\/em>Thus, it is hard, if not impossible, to explain the\u00a0surprise part of Simpson\u2019s reversal\u00a0without postulating that\u00a0human intuition is governed by causal calculus\u00a0together with a persistent tendency to attribute causal\u00a0interpretation to statistical associations.\u201d<\/p>\n 1.\u00a0In Simpson\u2019s paradox we witness the\u00a0simultaneous orderings: (a1+b1)\/(c1+d1)>\u00a0<\/em>(a2+b2)\/(c2+ Final Remarks<\/strong><\/p>\n I used to be extremely impatient with the slow pace in which\u00a0causal ideas have been penetrating scientific communities\u00a0that are not used to talk cause-and-effect.\u00a0Recently, however, I re-read Thomas Kuhn\u2019 classic\u00a0The Structure of Schientific Revolution<\/em> and\u00a0I found there a quote that made me calm, content,\u00a0even humorous and hopeful. Here it is:<\/p>\n \u2014\u2014\u2014\u2014\u2014- Kuhn\u00a0\u2014\u2014\u2014\u2014\u2014-<\/p>\n \u201cThe transfer of allegiance from paradigm to paradigm is a\u00a0conversion experience that cannot be forced. Lifelong\u00a0resistance, particularly from those whose productive careers\u00a0have committed them to an older tradition of normal science,\u00a0is not a violation of scientific standards but an index\u00a0to the nature of scientific research itself.\u201d \u201cConversions will occur a few at a time until, after the last\u00a0holdouts have died, the whole profession will again be\u00a0practicing under a single, but now a different, paradigm.\u201d We are now seeing the last holdouts.<\/p>\n Cheers,<\/p>\n Judea<\/p>\n Addendum: Simpson and the Potential-Outcome Camp<\/strong><\/span><\/p>\n My discussion of the four Simpson\u2019s papers would be incomplete without mentioning another paper, which represents the thinking within the potential outcome camp. The paper in question is \u201cA Fruitful Resolution to Simpson\u2019s Paradox via Multiresolution Inference,\u201d by Keli Liu and Xiao-Li Meng (2014), http:\/\/www.stat.columbia.edu\/~gelman\/stuff_for_blog\/LiuMengTASv2.pdf<\/a>\u00a0which appeared in the same issue of Statistical Science as my \u201cUnderstanding Simpson\u2019s Paradox\u201d http:\/\/ftp.cs.ucla.edu\/pub\/stat_ser\/r414-reprint.pdf<\/a>.<\/p>\n The intriguing feature of Liu and Meng\u2019s paper is that they, too, do not see any connection to causality. In their words: \u201cPeeling away the [Simpson\u2019s] paradox is as easy (or hard) as avoiding a comparison of apples and oranges, a concept requiring no mention of causality\u201d p.17, and again: \u201d The central issues of Simpson\u2019s paradox can be addressed adequately without necessarily invoking causality.\u201d (p. 18). Two comments:<\/p>\n Thus, in contrast to the data-only economists (Spanos), the potential-outcome camp does not object to causal reasoning per-se, this is their specialty. What they object to are attempts to resolve Simpson\u2019s paradox formally and completely, namely, explicate formally what the differences are between \u201capples and oranges\u201d and deal squarely with the decision problem: \u201cWhat to do in case of reversal.\u201d<\/p>\n Why are they resisting the complete solution? Because (and this is a speculation) the complete solution requires graphical tools and we all know the attitude of potential-outcome enthusiasts towards graphs. We dealt with this cultural peculiarity before so, at this point, we should just add Simpson\u2019s paradox to their list of challenges, and resign humbly to the slow pace with which Kuhn\u2019s paradigms are shifting.<\/p>\n Judea<\/p>\n","protected":false},"excerpt":{"rendered":" Contributor: Judea Pearl If you search Google for \u201cSimpson\u2019s paradox,\u201d as I\u00a0did yesterday, you will get 111,000 results, more than any other\u00a0statistical paradox that I could name. What elevates\u00a0this innocent\u00a0reversal of association to \u201cparadoxical\u201d status, and why it has\u00a0captured the fascination of statisticians, mathematicians\u00a0and philosophers for over a century are questions\u00a0that we discussed at length […]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[35],"tags":[],"class_list":["post-1688","post","type-post","status-publish","format-standard","hentry","category-simpsons-paradox"],"_links":{"self":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1688","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=1688"}],"version-history":[{"count":49,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1688\/revisions"}],"predecessor-version":[{"id":1869,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1688\/revisions\/1869"}],"wp:attachment":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=1688"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=1688"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=1688"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nMalinas, G. and Bigelow, J. \u201cSimpson\u2019s Paradox,\u201d\u00a0The Stanford\u00a0Encyclopedia of Philosophy (Summer\u00a02016 Edition)<\/em>, Edward N. Zalta (ed.), URL = <http:\/\/plato.stanford.edu\/
\nSpanos, A., \u201cRevisiting Simpson\u2019s Paradox: a statistical\u00a0misspecification perspective,\u201d ResearchGate Article,\u00a0<https:\/\/www.
\n<http:\/\/arxiv.org\/pdf\/1605.
\nMemetea, S. \u201cSimpson\u2019s Paradox in Epistemology and Decision Theory,\u201d The University\u00a0of British Columbia (Vancouver), Department of Philosophy, Ph.D. Thesis, May 2015.
\nhttps:\/\/open.library.ubc.ca\/
\nBandyopadhyay, P.S., Raghavan, R.V., Deruz, D.W., and Brittan, Jr., G.\u00a0\u201cTruths about Simpson\u2019s Paradox Saving the Paradox from Falsity,\u201d in\u00a0Mohua Banerjee and Shankara Narayanan Krishna (Eds.),\u00a0Logic and Its Applications, Proceedings of the 6th Indian Conference\u00a0ICLA 2015<\/em>\u00a0, LNCS 8923, Berlin Heidelberg: Springer-Verlag, pp. 58-73,\u00a02015 .
\nhttps:\/\/www.academia.edu\/
\n2.\u00a0The no-change provision is probabilistic; it permits\u00a0the action to change the classification of individual units\u00a0so long as the relative sizes of the subpopulations remain unaltered.<\/span><\/p>\n
\n
\np. 151<\/p>\n
\np. 152<\/p>\n
\n\n