{"id":1631,"date":"2016-06-20T07:45:08","date_gmt":"2016-06-20T07:45:08","guid":{"rendered":"http:\/\/causality.cs.ucla.edu\/blog\/?p=1631"},"modified":"2016-06-20T21:02:42","modified_gmt":"2016-06-20T21:02:42","slug":"recollections-from-the-wce-conference-at-stanford","status":"publish","type":"post","link":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/2016\/06\/20\/recollections-from-the-wce-conference-at-stanford\/","title":{"rendered":"Recollections from the WCE conference at Stanford"},"content":{"rendered":"

On May 21, Kosuke Imai and I participated in a panel on Mediation,\u00a0at the annual meeting of the West Coast Experiment\u00a0Conference, organized by Stanford Graduate School of\u00a0Business\u00a0http:\/\/www.gsb.stanford.edu\/facseminars\/conferences\/west-coast-experiments-conference<\/a>.\u00a0The following are some of my recollections from that\u00a0panel.<\/p>\n

1.
\nWe began the discussion by reviewing causal mediation\u00a0analysis and summarizing the exchange we had on the\u00a0pages of Psychological Methods (2014)
\nhttp:\/\/ftp.cs.ucla.edu\/pub\/stat_ser\/r389-imai-etal-commentary-r421-reprint.pdf<\/a><\/p>\n

My slides for the panel can be viewed here:
\nhttp:\/\/web.cs.ucla.edu\/~kaoru\/stanford-may2016-bw.pdf<\/a><\/p>\n

We ended with a consensus regarding the importance of\u00a0causal mediation and the conditions for identifying of Natural\u00a0Direct and Indirect Effects, from randomized as well\u00a0as observational studies.<\/p>\n

2.
\nWe proceeded to discuss the symbiosis between the\u00a0structural and the counterfactual languages.\u00a0Here I focused on slides 4-6 (page 3), and remarked\u00a0that only those who are willing to solve a toy problem\u00a0from begining to end, using both potential outcomes and DAGs can\u00a0understand the tradeoff between the two.\u00a0Such a toy problem (and its solution) was presented\u00a0in slide 5 (page 3) titled “Formulating a problem\u00a0in Three Languages” and the questions that I asked the audience\u00a0are still ringing in my ears.\u00a0Please have a good look at these two sets of assumptions\u00a0and ask yourself:<\/p>\n

a. Have we forgotten any assumption?
\nb. Are these assumptions consistent?
\nc. Is any of the assumptions redundant\u00a0(i.e. does it follow logically from the others)?
\nd. Do they have testable implications?
\ne. Do these assumptions permit the identification\u00a0of causal effects?
\nf. Are these assumptions plausible in the context\u00a0of the scenario given?<\/p>\n

As I was discussing these questions over slide 5, the\u00a0audience seemed to be in general agreement with the\u00a0conclusion that, despite their logical equivalence,\u00a0the graphical language\u00a0 enables\u00a0 us to answer these\u00a0questions immediately while the potential outcome language\u00a0remains silent on all.<\/p>\n

I consider this example to be pivotal to the comparison of the two frameworks.\u00a0I hope that questions a,b,c,d,e,f will be remembered,\u00a0and speakers from both camps will be asked to\u00a0address them squarely and explicitly .<\/p>\n

The fact that graduate students made up the majority\u00a0of the participants gives me the hope that\u00a0questions a,b,c,d,e,f will finally receive the\u00a0attention they deserve.<\/p>\n

3.
\nAs we discussed the virtues of graphs, I found it necessary to\u00a0reiterate the observation that DAGs are more than just\u00a0“natural and convenient way to express assumptions about causal\u00a0structures” (Imbens and Rubin , 2013, p. 25).\u00a0Praising their transparency while ignoring their inferential\u00a0power misses the main role that graphs play in\u00a0causal analysis. The power of graphs lies in\u00a0computing complex implications of causal assumptions\u00a0(i.e., the “science”) no matter in what language\u00a0they are expressed.\u00a0 Typical implications are: conditional\u00a0independencies among variables and counterfactuals, what covariates\u00a0need be controlled to remove confounding or selection bias,\u00a0whether effects can be identified, and more.\u00a0These implications could, in principle, be derived from\u00a0any equivalent representation of the causal assumption,\u00a0not necessarily graphical, but not before incurring\u00a0a prohibitive computational cost. See, for example,\u00a0what happens when economists try to replace\u00a0d-separation with graphoid axioms\u00a0http:\/\/ftp.cs.ucla.edu\/pub\/stat_ser\/r420.pdf<\/a>.<\/p>\n

4.
\nFollowing the discussion of representations, we\u00a0addressed questions posed to us by the audience,\u00a0in particular, five questions submitted by Professor\u00a0Jon Krosnick (Political Science, Stanford).<\/p>\n

I summarize them in the following slide:<\/p>\n

Krosnick’s Questions to Panel
\n———————————————-
\n1) Do you think an experiment has any value without mediational analysis?
\n2) Is a separate study directly manipulating the mediator useful?\u00a0How is the second study any different from the first one?
\n3) Imai’s correlated residuals test seems valuable for distinguishing\u00a0fake from genuine mediation. Is that so?\u00a0And how it is related to traditional mediational test?
\n4) Why isn’t it easy to test whether participants who show the largest increases in the\u00a0posited mediator show the largest changes in the outcome?
\n5) Why is mediational analysis\u00a0any “worse” than any other method of investigation?
\n———————————————-
\nMy answers focused on question 2, 4 and 5,\u00a0which I summarize below:<\/p>\n

2)
\nQ. Is a separate study directly manipulating the mediator useful?
\nAnswer: Yes, it is useful if physically feasible but, still,\u00a0it cannot give us an answer to the basic mediation question:\u00a0“What percentage of the observed response is due to\u00a0mediation?”\u00a0The concept of mediation is necessarily counterfactual,\u00a0i.e. sitting on the top layer of the causal hierarchy\u00a0(see “Causality” chapter 1). It cannot be defined therefore\u00a0in terms of population experiments, however clever.\u00a0Mediation can be evaluated with the help of counterfactual\u00a0assumptions such as “conditional ignorability”\u00a0or “no interaction,” but these assumptions cannot be verified in\u00a0population experiments.<\/p>\n

4)
\nQ. Why isn’t it easy to test whether participants who show\u00a0the largest increases in the\u00a0posited mediator show the largest changes in the outcome?
\nAnswer:\u00a0Translating the question to counterfactual notation\u00a0the test suggested requires the existence of\u00a0monotonic function f_m such that, for every individual,\u00a0we have\u00a0Y_1 – Y_0 =f_m (M_1 – M_0)<\/p>\n

This condition expresses a feature we expect to find\u00a0in mediation, but it cannot be taken as a DEFINITION\u00a0of mediation. This condition is essentially the way\u00a0indirect effects are defined in the\u00a0Principal Strata framework (Frangakis and Rubin, 2002)\u00a0the deficiencies of which are well known.\u00a0See http:\/\/ftp.cs.ucla.edu\/pub\/stat_ser\/r382.pdf<\/a>.<\/p>\n

In particular, imagine a switch S controlling two\u00a0light bulbs L1 and L2. Positive correlation between\u00a0L1 and L2 does not mean that L1 mediates between the\u00a0switch and L2. Many examples of incompatibility\u00a0are demonstrated in the paper above.<\/p>\n

The conventional mediation tests (in the Baron and Kenny\u00a0tradition) suffer from the same problem; they test\u00a0features of mediation that are common in linear systems,\u00a0but not the essence of mediation which is universal\u00a0to all systems, linear and nonlinear,\u00a0continuous as well as categorical variables.<\/p>\n

5)
\nQ. Why is mediational analysis\u00a0any “worse” than any other method of investigation?
\nAnswer:\u00a0The answer is closely related to the one given to\u00a0question 3).\u00a0Mediation is not a “method” but a property of the\u00a0population which is defined counterfactually,\u00a0and therefore requires counterfactual assumption\u00a0for evaluation. Experiments are not sufficient; and\u00a0in this sense mediation is “worse” than other properties\u00a0under investigation, eg., causal effects, which can\u00a0be estimated entirely from experiments.<\/p>\n

About the only thing we can ascertain experimentally\u00a0is whether the (controlled) direct effect differs from the\u00a0total effect, but we cannot\u00a0evaluate the extent of mediation.<\/p>\n

Another way to appreciate why stronger assumptions are needed\u00a0for mediation is to note that non-confoundedness is not\u00a0the same as ignorability. For non-binary variables\u00a0one can construct examples where X and Y are not confounded\u00a0( i.e., P(y|do(x))= P(y|x)) and yet they are not ignorable,\u00a0(i.e., Y_x is not independent of X.)\u00a0Mediation requires ignorability in addition to\u00a0nonconfoundedness.<\/p>\n

Summary
\nOverall, the panel was illuminating, primarily\u00a0due to the active participation of curious students.\u00a0It gave me good reasons to believe that Political Science is\u00a0destined to become a bastion of modern causal analysis.\u00a0I wish economists would follow suit, despite the hurdles\u00a0they face in getting causal analysis to economics education.
\nhttp:\/\/ftp.cs.ucla.edu\/pub\/stat_ser\/r391.pdf<\/a>
\nhttp:\/\/ftp.cs.ucla.edu\/pub\/stat_ser\/r395.pdf<\/a><\/p>\n

Judea<\/p>\n","protected":false},"excerpt":{"rendered":"

On May 21, Kosuke Imai and I participated in a panel on Mediation,\u00a0at the annual meeting of the West Coast Experiment\u00a0Conference, organized by Stanford Graduate School of\u00a0Business\u00a0http:\/\/www.gsb.stanford.edu\/facseminars\/conferences\/west-coast-experiments-conference.\u00a0The following are some of my recollections from that\u00a0panel. 1. We began the discussion by reviewing causal mediation\u00a0analysis and summarizing the exchange we had on the\u00a0pages of Psychological Methods […]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,16,26,37],"tags":[],"class_list":["post-1631","post","type-post","status-publish","format-standard","hentry","category-counterfactual","category-general","category-mediated-effects","category-structural-equations"],"_links":{"self":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1631","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=1631"}],"version-history":[{"count":8,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1631\/revisions"}],"predecessor-version":[{"id":1640,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1631\/revisions\/1640"}],"wp:attachment":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=1631"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=1631"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=1631"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}