{"id":2079,"date":"2019-12-11T20:44:21","date_gmt":"2019-12-11T20:44:21","guid":{"rendered":"http:\/\/causality.cs.ucla.edu\/blog\/?p=2079"},"modified":"2020-09-03T23:34:54","modified_gmt":"2020-09-03T23:34:54","slug":"generalizing-experimental-results-by-leveraging-knowledge-of-mechanisms","status":"publish","type":"post","link":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/2019\/12\/11\/generalizing-experimental-results-by-leveraging-knowledge-of-mechanisms\/","title":{"rendered":"Generalizing Experimental Results by Leveraging Knowledge of Mechanisms"},"content":{"rendered":"\n<p>In a <a class=\"PostLinkPreviewWithPost-link\" href=\"https:\/\/www.lesswrong.com\/posts\/wwbrvumMWhDfeo652\">recent post (and papers)<\/a>, Anders\u00a0<span style=\"font-size: inherit;\">Huitfeldt\u00a0<\/span><span style=\"font-size: inherit;\">and co-authors have discussed ways of achieving external validity in the presence of \u201ceffect heterogeneity.\u201d\u00a0These results are not immediately inferable using a standard (non-parametric) selection diagram, which has led them to conclude that selection diagrams may not be helpful for\u00a0 &#8220;thinking more closely\u00a0about effect heterogeneity&#8221; and, thus, might be &#8220;throwing the baby\u00a0out with the bathwater.&#8221;<\/span><\/p>\n<p>Taking a closer look at the analysis of Anders and co-authors,\u00a0and using their very same examples, we came to quite different conclusions. In those cases, transportability is not immediately inferable in a fully nonparametric structural model for a simple reason:\u00a0it relies on <em>functional constraints<\/em> on the structural equation of the outcome. Once these constraints are properly incorporated in the analysis, all results flow naturally from the structural model,\u00a0and selection diagrams prove to be indispensable for thinking about heterogeneity, for extrapolating results across populations, and for protecting analysts from unwarranted generalizations.\u00a0 <a href=\"https:\/\/ftp.cs.ucla.edu\/pub\/stat_ser\/r492.pdf\">See details in the<\/a><a href=\"http:\/\/causality.cs.ucla.edu\/blog\/wp-content\/uploads\/2019\/12\/cinelli_pearl_generalizing_experimental_results.pdf\"> full<\/a><a href=\"http:\/\/causality.cs.ucla.edu\/blog\/wp-content\/uploads\/2019\/12\/cinelli_pearl_generalizing_experimental_results.pdf\"> note<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In a recent post (and papers), Anders\u00a0Huitfeldt\u00a0and co-authors have discussed ways of achieving external validity in the presence of \u201ceffect heterogeneity.\u201d\u00a0These results are not immediately inferable using a standard (non-parametric) selection diagram, which has led them to conclude that selection diagrams may not be helpful for\u00a0 &#8220;thinking more closely\u00a0about effect heterogeneity&#8221; and, thus, might be [&hellip;]<\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[41,17,18],"tags":[56,55,57,54],"class_list":["post-2079","post","type-post","status-publish","format-standard","hentry","category-data-fusion","category-generalizability","category-identification","tag-functional-constraints","tag-generalizability","tag-mechanisms","tag-transportability"],"_links":{"self":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/2079","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=2079"}],"version-history":[{"count":4,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/2079\/revisions"}],"predecessor-version":[{"id":2299,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/2079\/revisions\/2299"}],"wp:attachment":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=2079"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=2079"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=2079"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}