{"id":2024,"date":"2019-08-14T23:26:46","date_gmt":"2019-08-14T23:26:46","guid":{"rendered":"http:\/\/causality.cs.ucla.edu\/blog\/?p=2024"},"modified":"2021-03-18T16:10:55","modified_gmt":"2021-03-18T16:10:55","slug":"a-crash-course-in-good-and-bad-control","status":"publish","type":"post","link":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/2019\/08\/14\/a-crash-course-in-good-and-bad-control\/","title":{"rendered":"A Crash Course in Good and Bad Control"},"content":{"rendered":"

Carlos Cinelli, Andrew Forney and Judea Pearl<\/strong><\/em><\/p>\n

Update: check the updated and extended version of the crash course here.<\/strong><\/em><\/span><\/span><\/a><\/p>\n

Introduction<\/strong><\/h2>\n

If you were trained in traditional regression pedagogy, chances are that you have heard about the problem of “bad controls”. The problem arises when we need to decide whether the addition of a variable to a regression equation helps getting estimates closer to the parameter of interest. Analysts have long known that some variables, when added to the regression equation, can produce unintended discrepancies between the regression coefficient and the effect that the coefficient is expected to represent. Such variables have become known as “bad controls”, to be distinguished from “good controls” (also known as “confounders” or “deconfounders”) which are variables that must be added to the regression equation to eliminate what came to be known as “omitted variable bias” (OVB).<\/span><\/p>\n

Recent advances in graphical models have produced a simple criterion to distinguish good from bad controls, and the purpose of this note is to provide practicing analysts a concise and visible summary of this criterion through illustrative examples. We will assume that readers are familiar with the notions of “path-blocking” (or d-separation) and back-door paths. For a gentle introduction, see<\/span> d-Separation without Tears<\/span><\/i><\/a>.\u00a0<\/span><\/p>\n

In the following set of models,\u00a0 the target of the analysis is the average causal effect (ACE) of a treatment X on an outcome Y, which stands for the expected increase of Y per unit of a controlled increase in X. Observed variables will be designated by black dots and unobserved variables by white empty circles. Variable Z (highlighted in red) will represent the variable whose inclusion in the regression is to be decided, with “good control” standing for <\/span>bias reduction<\/span><\/i>, “bad control” standing for <\/span>bias increase<\/span><\/i> and \u201cnetral control\u201d when the addition of Z <\/span>does not increase nor reduce bias<\/span><\/i>. For this last case, we will also make a brief remark about how Z could affect the <\/span>precision<\/span><\/i> of the ACE estimate.<\/span><\/p>\n

Models<\/strong><\/h2>\n

Models 1, 2 and 3 \u2013 Good Controls\u00a0<\/i><\/strong><\/p>\n

\"\"<\/a> \"\"<\/a> \"\"<\/a><\/p>\n

In model 1,\u00a0 Z stands for a common cause of both X and Y. Once we control for Z, we block the back-door path from X to Y, producing an unbiased estimate of the ACE.<\/span>\u00a0<\/span><\/i><\/p>\n

In models 2 and 3, Z is not a common cause of both X and Y, and therefore, not a traditional \u201cconfounder\u201d as in model 1. Nevertheless, controlling for Z blocks the back-door path from X to Y due to the unobserved confounder U, and again, produces an unbiased estimate of the ACE.<\/span><\/p>\n

Models 4, 5 and 6 – Good Controls<\/i><\/strong><\/p>\n

\"\"<\/a> \"\"<\/a> \"\"<\/a><\/p>\n

When thinking about possible threats of confounding, one needs to keep in mind that common causes of X and any <\/span>mediator<\/span><\/i> (between X and Y) also confound the effect of X on Y. Therefore, models 4, 5 and 6 are analogous to models 1, 2 and 3 — controlling for Z blocks the backdoor path from X to Y and produces an unbiased estimate of the ACE.<\/span><\/p>\n

Model 7 – Bad Control<\/i><\/strong><\/p>\n

\"\"<\/a><\/p>\n

We now encounter our first \u201cbad control\u201d. Here Z is correlated with the treatment and the outcome and it is also a \u201cpre-treatment\u201d variable. Traditional econometrics textbooks would deem Z a \u201cgood control\u201d. The backdoor criterion, however, reveals that Z is a \u201cbad control\u201d. Controlling for Z will <\/span>induce bias<\/span><\/i> by opening the backdoor path X \u2190 U<\/span>1<\/span><\/sub>\u2192 Z\u2190 U<\/span>2<\/span><\/sub>\u2192Y, thus spoiling a previously unbiased estimate of the ACE.<\/span><\/p>\n

Model 8 – Neutral Control (possibly good for precision)<\/i><\/strong><\/p>\n

\"\"<\/a><\/p>\n

Here Z is not a confounder nor does it block any backdoor paths. Likewise, controlling for Z does not open any backdoor paths from X to Y. Thus, in terms of <\/span>bias<\/span><\/i>, Z is a \u201cneutral control\u201d. Analysis shows, however, that controlling for Z <\/span>reduces the variation of the outcome<\/span><\/i> variable Y<\/span><\/i>, and helps improve the <\/span>precision<\/span><\/i> of the ACE estimate in finite samples.<\/span><\/p>\n

Model 9 – Neutral control (possibly bad for precision)<\/i><\/strong><\/p>\n

\"\"<\/a><\/p>\n

Similar to the previous case, here Z is “neutral” in terms of bias reduction. However, controlling for Z <\/span>will reduce the variation of treatment variable X<\/span><\/i> and so may <\/span>hurt <\/span><\/i>the <\/span>precision<\/span><\/i> of the estimate of the ACE in finite samples.\u00a0\u00a0<\/span><\/p>\n

Model 10 – Bad control<\/i><\/strong><\/p>\n

\"\"<\/a><\/p>\n

We now encounter our second \u201cpre-treatment\u201d \u201cbad control\u201d, due to a phenomenon called \u201cbias amplification\u201d <\/span>(read more here)<\/span><\/a>. Naive control for Z in this model will not only fail to deconfound the effect of X on Y, but, in linear models, <\/span>will amplify any existing bias.<\/span><\/i><\/p>\n

Models 11 and 12 – Bad Controls<\/i><\/strong><\/p>\n

\"\"<\/a> \"\"<\/a><\/p>\n

If our target quantity is the ACE, we want to leave all channels through which the causal effect flows \u201cuntouched\u201d. <\/span><\/p>\n

In Model 11, Z is a mediator of the causal effect of X on Y. Controlling for Z will block the very effect we want to estimate, thus biasing our estimates.\u00a0 <\/span><\/p>\n

In Model 12, although Z is not itself a mediator of the causal effect of X on Y, controlling for Z is equivalent to partially controlling for the mediator M, and will thus bias our estimates. <\/span><\/p>\n

Models 11 and 12 violate the backdoor criterion, which excludes controls that are descendants of the treatment along paths to the outcome.<\/span><\/p>\n

Model 13 – Neutral control (possibly good for precision)<\/i><\/strong><\/p>\n

\"\"<\/a><\/p>\n

At first look, model 13 might seem similar to model 12, and one may think that adjusting for Z would bias the effect estimate, by restricting variations of the mediator M. However, the key difference here is that Z is a <\/span>cause, not an effect, <\/span><\/i>of the mediator (and, consequently, also a cause of Y). Thus, model 13 is analogous to model 8, and so controlling for Z will be neutral in terms of bias and may increase precision of the ACE estimate in finite samples.<\/span><\/p>\n

Model 14 – Neutral controls (possibly helpful in the case of selection bias)<\/i><\/strong><\/p>\n

\"\"<\/a> \"\"<\/a><\/p>\n

Contrary to econometrics folklore, not all \u201cpost-treatment\u201d variables are inherently bad controls. In models 14 and 15 controlling for Z <\/span>does not<\/span><\/i> open any confounding paths between X and Y. Thus, Z is neutral in terms of bias. However, controlling for Z <\/span>does<\/span><\/i> reduce the variation of the treatment variable X and so may hurt the precision of the ACE estimate in finite samples. Additionally, in model 15, suppose one has only samples with W = 1 recorded (a case of selection bias). In this case, controlling for Z can help obtaining the W-specific effect of X on Y, by blocking the colliding path due to W.<\/span><\/p>\n

Model 16 – Bad control<\/i><\/strong><\/p>\n

\"\"<\/a><\/p>\n

Contrary to Models 14 and 15, here controlling for Z is no longer harmless, since it opens the backdoor path X \u2192 Z \u2190 U \u2192 Y and so biases the ACE.<\/span><\/p>\n

Model 17 – Bad Control<\/i><\/strong><\/p>\n

\"\"<\/a><\/p>\n

Here, Z is not a mediator, and one might surmise that, as in Model 14, controlling for Z is harmless. However, controlling for the effects of the outcome Y will induce bias in the estimate of the ACE, making Z a \u201cbad control\u201d. A visual explanation of this phenomenon using \u201cvirtual colliders\u201d can be <\/span>found here<\/span><\/a>.<\/span><\/p>\n

Model 17 is usually known as a \u201ccase-control bias\u201d or \u201cselection bias\u201d. Finally, although controlling for Z will generally bias<\/span> numerical estimates <\/span><\/i>of the ACE, it does have an exception when X has <\/span>no causal effect<\/span><\/i> on Y. In this scenario, X is still d-separated from Y even after conditioning on Z. Thus, adjusting for Z is valid for <\/span>testing<\/span><\/i> whether the effect of X on Y <\/span>is zero. <\/span><\/i><\/p>\n","protected":false},"excerpt":{"rendered":"

Carlos Cinelli, Andrew Forney and Judea Pearl Update: check the updated and extended version of the crash course here. Introduction If you were trained in traditional regression pedagogy, chances are that you have heard about the problem of “bad controls”. The problem arises when we need to decide whether the addition of a variable to […]<\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,47,48,13,18],"tags":[46,44,43,45],"class_list":["post-2024","post","type-post","status-publish","format-standard","hentry","category-back-door-criterion","category-bad-control","category-econometrics","category-economics","category-identification","tag-back-door-criterion","tag-bad-control","tag-confounding","tag-econometrics"],"_links":{"self":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/2024","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=2024"}],"version-history":[{"count":19,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/2024\/revisions"}],"predecessor-version":[{"id":2373,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/2024\/revisions\/2373"}],"wp:attachment":[{"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=2024"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=2024"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/causality.cs.ucla.edu\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=2024"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}