1.1. Implicit Associations

Implicit associations are sometimes portrayed as among the most significant discoveries by social psychologists over the past several decades. For our purposes, “associations” may be understood as traces of past experiences that encode evaluations of an object, including social objects. Such associations are often regarded as “biases” in that they encode favorable or unfavorable evaluations of social groups, including social groups defined in terms of race. There is no consensus on how to precisely characterize the sense(s) in which implicit biases are “implicit”. However, proposals include: being relatively inaccessible to conscious awareness or introspection (Greenwald, McGhee, & Schwartz 1998), being relatively difficult to bring under reflective control, and contributing to quick and efficient cognitive processing that is more or less “automatic” (Gawronski & De Houwer 2014). Similarly, there is no consensus on the ontology of implicit biases—whether they are a type of belief, attitude, feeling, process, motivation, psychological trait, or situational feature (see Brownstein, Madva & Gawronski 2019 and Holroyd, Scaife, & Stafford 2017 for discussion). Little in this article depends on the resolution of such disagreements over the nature of implicit bias, so I am noncommittal in what follows.⁴ This article places greater importance on whether the targets of implicit measures, whatever precisely they measure, plausibly have significant behavioral effects.

1.2. Measuring Implicit Bias

A variety of indirect tests have been developed to measure implicit bias, many of which involve timed sorting tasks between visual images or words on the one hand, and positive and negative attributes on the other. The best-known of these is the Implicit Association Test (IAT) (Greenwald et al. 1998), but other measures include The Affective Lexical Priming Score, The Go/No-Go Association Task, The Affect Misattribution Procedure, The Sorting Paired Feature Task, and the Multi-Category Implicit Association Test (see Bar-Anan & Nosek 2014 for discussion). A guiding idea behind many of these indirect tests for implicit associations to social groups is that greater cognitive effort and time is required to match a positive attribute to a social group, or to a member of a social group, when subjects implicitly associate negative attributes with that group, and vice versa. Thus, for several of the above, greater response times and error rates on sorting tasks involving positive and negative attributes are interpreted to measure the strength of implicit associations that may encode biases.

Indirect tests of implicit attitudes contrast with direct tests of subjects’ explicit attitudes, which typically rely on their self-reports about the relevant attitude. The results of the two testing methods often do not coincide—for example, Hofmann, Gawronski, Gschwendner, Le, and Schmitt (2005: 1376) estimate an average correlation of r = .24 between explicit and implicit measures, with correlations varying significantly across contexts. Based in part on the relatively low correlation between direct and indirect testing results, many social psychologists differentiate implicit from explicit cognition. However, there is no consensus on the extent to which implicit and explicit cognition are distinct or stable features of the human mind,⁵ and I will remain neutral on this issue in what follows. The central questions about implicit racial bias this article investigates may be formulated in terms of implicit measures, viz.: What predictive relations hold between implicit measures of racial bias and racially biased behaviors; and, are the behavioral effects associated with measures of implicit racial bias significant?

One strategy for answering both questions is by consulting meta-analyses that synthesize the findings of large numbers of studies. This strategy has been widely adopted in discussions on the significance of implicit bias research even though some philosophers of science have discounted the general evidential value of meta-analytic studies. For example, some philosophers have argued meta-analyses merely aggregate potentially biased results (Romero 2016) or merely highlight the analysts’ subjective and arbitrary decisions (Stegenga 2011). It is beyond the scope of this article to rebut these general critiques, and with minor exceptions I do not engage them. Instead, this article’s investigations proceed on the assumption that meta-analyses are among our best sources of evidence for stochastic effects on large populations (Holman 2019; Bruner & Holman 2019).

2.1. A Series of Competing Meta-Analyses

The first meta-analysis of studies on implicit measures and behaviors, Greenwald, Poehlman, Uhlmann, and Banaji (2009), encompassed many behavioral domains, including consumer choices, voting, and “intergroup” behaviors, that is, behaviors toward (members of) social groups. According to this meta-analysis, the most common measure of implicit bias—the IAT—could predict, in some contexts better than explicit measures, a variety of behaviors, especially socially sensitive and spontaneous intergroup behaviors involving race and ethnicity (Greenwald et al. 2009: 28). The mean correlations between IAT scores and racial and other intergroup behaviors was, according to Greenwald et al.’s estimate, r = .24 and r = .20 respectively (2009: 28).⁷ A second meta-analysis, Cameron, Brown-Iannuzzi, and Payne (2012), estimated an average correlation of r = .28 between sequential priming measures of implicit social cognition (rather than the IAT) and social behaviors. Both meta-analyses supported the validity of implicit measures as predictive tools and potentially implicated implicit racial bias in discriminatory behaviors in a variety of social contexts, including hiring, admissions, healthcare, law enforcement, and other daily interactions.

Subsequent meta-analyses, however, appeared to weaken those conclusions. Instead of correlations in the range of r = .20 to .28, Oswald, Mitchell, Blanton, Jaccard, and Tetlock (2013: 178) estimated a smaller mean correlation of r = .14 between IAT scores and behaviors toward individuals or groups defined in terms of race or ethnicity. This estimate was derived by employing advances in statistical techniques⁸ and adopting different inclusion criteria for measured behavioral effects (see Section 2.3 for discussion of these criteria). According to this meta-analysis, the IAT did not add any predictive power to explicit measures for predicting behavior, and in particular was not usefully predictive of spontaneous behaviors (2013: 183). Oswald et al. concluded, “the IAT provides little insight into who will discriminate against whom, and provides no more insight than explicit measures of bias” (2013: 188).

These substantially weaker results sparked controversy over the statistical and societal significance of correlations linking implicit measures to biased behaviors. For example, Machery (2016), drawing on Oswald et al.’s (2013) meta-analysis, urged researchers and others to significantly pare back their estimation of implicit bias’s role in explaining discrimination and inequality:

one may get the erroneous impression that indirect measures are excellent predictors of biased behaviors, since implicit attitudes are called upon to explain many social ills. The truth, however, is quite different … since the predictive validity of the implicit association test is extremely low. (Machery 2016: 119)

Research that immediately followed Machery’s appraisal appeared to buttress it. For example, Carlsson and Agerström (2016) re-analyzed the same set of studies as Oswald et al. (2013) but operationalized behavioral effects more strictly so as to include only genuinely discriminatory behavioral effects.⁹ They reported a similar estimate to Oswald et al. (2013) of the mean correlation between implicit measures and behavioral effects, viz. r = .15. More strikingly, Forscher et al. reported “the overall correlation between implicit measures and behavior in our meta-analysis was small… (r = .09)” (2019: 41).¹⁰ In concert with this trend of diminishing correlations and effect sizes, more strident criticisms of implicit bias research proliferated (e.g., Mitchell & Tetlock 2017; Singal 2017; Bartlett 2017; Mitchell 2018).

2.2. Assessing Competing Meta-Analyses

Authors of competing meta-analyses of studies on implicit measures and behaviors have been divided over two central controversies—one methodological, the other interpretive (cf. Greenwald et al. 2015 and Oswald et al. 2015 for discussion). The methodological controversy is over whether to include, in meta-analytic estimates of a mean correlation between implicit measures and behaviors, behavioral effects that a study’s authors did not predict would be correlated with implicit measures. The interpretive controversy is over whether the various mean correlations estimated by competing meta-analyses, regardless of their methodology, suggest implicit biases have significant discriminatory effects. In what follows, I argue the methodological controversy is inconclusive in the sense that both methodological choices—whether to include or exclude behavioral effects based on author prediction—are equally warranted given the available evidence. With respect to the interpretive controversy, I maintain defenders of implicit bias research have substantially stronger arguments for the conclusion that the mean correlations estimated by competing meta-analyses suggest implicit biases have significant discriminatory effects.

2.3. The Methodological Controversy: Meta-Analytic Inclusion Criteria for Behavioral Effects

Greenwald et al. (2009) and Cameron et al. (2012) (as well as Kurdi et al. 2018; see Section 3) adopt inclusion criteria for behavioral effects that are sensitive to a study’s stated predictions; in contrast, Oswald et al. (2013), Carlsson and Agerström (2016), and Forscher et al. (2019) adopt inclusion criteria for behavioral effects that are insensitive to prediction. In other words, the latter but not the former group of meta-analyses include behavioral effects that the studies’ authors did not predict would be correlated to measures of implicit associations. According to Greenwald et al. (2015: 556), this methodological difference accounts for a “substantial portion” of the difference between the estimated mean correlation between implicit measures and behavioral effects by Oswald et al. (2013) (r = .14) and Greenwald et al. (2009) (r = .22).¹¹

Greenwald et al. (2015) defends Greenwald et al. (2009)’s (and Kurdi et al. 2018’s)¹² inclusion criteria on the grounds that authors’ expected findings of behavioral effects were often based on such plausible and mundane background hypotheses as:

(a) measures of attitude toward a group should predict behavior favorable or unfavorable to the group and (b) measures of a stereotype of the group should predict stereotype-consistent judgments or behavior toward members of that group (2015: 554).

On the assumption that such background hypotheses systematically informed prediction of behavioral effects in the studies analyzed, one might reasonably maintain that including measures of behavioral effects that were not predicted to occur—including some studies in which behavioral effects were correctly predicted to be absent—artificially reduces estimates of mean correlations between implicit measures and behaviors. Thus, there are potential benefits associated with the more restrictive inclusion criteria for behavioral effects based on author prediction adopted by Greenwald et al. (2009) (and Kurdi et al. 2018).

But this methodological choice also has potential costs. For example, Oswald et al. (2015) worries that excluding measures of behavioral effects based on researchers’ predictions can lead to significant information loss, particularly in light of inconsistent theoretical perspectives across studies that guide such prediction:

If the meta-analyst … defers to the judgment of different researchers in different research reports, inconsistency and the omission of substantial amounts of information can result. This possibility was a specific concern for us, because researchers conducting different IAT studies sometimes embraced different theories (2015: 563).

And indeed, predictions in the studies analyzed by both Oswald et al. (2013) and Greenwald et al. (2009) were guided by hypotheses based on single-association and double-dissociation models, on expectations based on the social sensitivity of the attitude measured, as well as on the spontaneity of the relevant behavior. Given this heterogeneity of theoretical perspectives, one might reasonably adopt a uniform policy on the inclusion of behavioral effects that is insensitive to author prediction, which, as Greenwald et al. (2015: 557) allows, “fits with … well-known methodological strategy” in psychological research. Moreover, such inclusion criteria might, arguably, render meta-analytic findings less susceptible to the analysts’ bias and less likely to amplify such biases as p-hacking and HARKing¹³ by the authors of the studies analyzed. On the basis of such potential strengths, Oswald et al. call their methodological choice less “problematic” (2015: 563) than the alternative inclusion criteria adopted by Greenwald et al. (2009) (and Kurdi et al. 2018).

By contrast, in light of the potential benefits and costs associated with both inclusion criteria, Greenwald et al. (2015: 556) judge that “both strategies were justifiable”. Given the evidence and arguments presented by critics and defenders of implicit bias research, this tolerant judgment is surely correct. Both methodologies have potential benefits, and neither has costs that demonstrably outweigh the benefits. Perhaps future inquiry will provide a basis for preferring either methodology. For example, future analysis might quantify the extent to which prediction in the relevant studies of implicit measures and behaviors were guided by plausible and relatively mundane background assumptions as opposed to inconsistent theoretical speculation. But even if such an analysis were performed, it might well be that no clear case would emerge for regarding either strategy as superior. Methodological controversies can be difficult to resolve, even in principle. Distinct methodologies can simply yield different kinds of information that are useful in different contexts and for different purposes.

The inconclusive nature of this methodological controversy might appear to support Stegenga (2011)’s view that competing meta-analyses typically highlight different, arbitrary choices by the analysts. But I believe we may draw a more informative conclusion by scrutinizing the methodologies of competing meta-analyses of studies on implicit measures and behaviors. In particular, it is informative that multiple meta-analyses of studies on implicit measures and behaviors with different but equally warranted methodologies all find correlations and accompanying effect sizes that suggest implicit biases have significant discriminatory effects in at least some circumstances. I argue for this conclusion in the next two subsections.

2.4. The Interpretive Controversy: Interpreting Correlations and Effect Sizes in Implicit Bias Research

Based on Oswald et al.’s (2013) finding of an average correlation of r = .14 between implicit measures and behaviors involving race and ethnicity, Machery (2016) characterizes the IAT’s predictive validity for behavioral effects as “extremely low” and Oswald et al. (2015: 565) calls this mean correlation “small by conventional standards” and “small (or very small)” (2015: 562). By contrast, Greenwald et al. (2015: 553) maintains this same “estimated aggregate correlational effect size… [is] large enough to explain discriminatory impacts that are societally significant”. This raises a question about how to interpret the magnitudes of correlation coefficients and accompanying effect sizes estimated by competing meta-analyses of studies on implicit measures and behaviors.

One strategy for answering is by comparison to other findings. Some comparisons appear favorable to defenders of implicit bias research, since even the lower end of meta-analytic estimates of mean correlations between implicit measures and behaviors are of similar magnitude to other findings in psychology that are considered significant, and consistent with causal relationships regarded as important in other domains. For example, ibuprofen’s mean correlation to reducing headache pain has been estimated as r = .14 (Meyer et al. 2001: 131). Similarly-sized correlations accompany a variety of psychological findings also regarded as well-established and significant. According to Richard, Bond, and Stokes-Zoota’s (2003) calculations, the mean correlation for the finding that people attribute failures to their bad luck is r = .10, that scarcity raises the perceived value of a commodity is r = .12, and that informational sources viewed as more credible are more persuasive is r = .10 (2003: 354–55). In light of these comparisons (see also Brownstein, Madva, & Gawronski 2020 for discussion), we should reject the assumption that a mean correlation of r = .14 between implicit measures and behaviors necessarily indicates trivial effects. Such an assumption is inconsistent with a variety of results regarded as significant in psychology and other scientific domains.

On the other hand, alternate comparisons are less favorable to defenders of implicit bias research. For example, the mean correlations between measures of general mental ability and job performance range from r = .28 to r = .38, depending on how much training the job requires (Schmidt & Hunter 2004: 165). Similarly, the mean correlation for the finding that people are more likely to be aggressive when provoked is r = .36 (Richard et al. 2003: 354) and the correlation between IQ scores for young adults (aged 19–23) and occupation type is r = .37 (Strenze 2007: 413). In comparison to these findings, measures of implicit bias fall short. Which comparisons are in fact appropriate benchmarks for judging the correlations and accompanying effect sizes in implicit bias research? By itself, invoking small numbers of comparisons is insufficient to resolve the interpretive controversy. Instead, we might turn to selecting an appropriate, general framework for interpreting the magnitude of correlations and effect sizes in psychological research, including those reported by competing meta-analyses on studies of implicit measures and behaviors.

A standard interpretational framework, widely adopted across the social sciences, is based on guidelines provided by the influential statistician and behavioral science researcher Jacob Cohen (Cohen 1988). Cohen’s guidelines are intended to help assign meaning to correlations and effect sizes in many contexts of human behavioral research, and may appear to favor critics of implicit bias research. According to Cohen’s guidelines, correlations of r = .5 and above should be regarded as “large”; correlations between r = .3 and r = .5 should be regarded as “medium”; and correlations from r = .1 to r = .3 should be interpreted as “small,” with correlations below r = .1 shading into insignificance. Oswald et al.’s (2015: 565) reference to a correlation of r = .14 as “small by conventional standards” likely reflects Cohen’s guidelines or a similar framework. These conventional standards imply that the mean correlations between implicit measures and behaviors, as estimated by Oswald et al. (2013), Carlsson and Agerström (2016) and Forscher et al. (2019), were nearing or had crossed the threshold into triviality.

Cohen’s guidelines, however, were intended only as rules of thumb whose appropriate application varies by context. In presenting them, Cohen noted the “risk inherent in offering conventional operational definitions … in as diverse a field of inquiry as behavioral science” (1988: 25). He further cautioned that “ ‘small,’ ‘medium,’ and ‘large’ are relative, not only to each other, but to the area of behavioral science or even more particularly to the specific content and research method being employed in any given investigation” (1988: 25). Arguably, Cohen’s guidelines are often less apposite than an alternate framework for interpreting the correlations and effect sizes found in research on implicit racial biases, since implicit racial biases’ effects, if they exist, plausibly range over a large number of events, affecting either large populations or single individuals over long time periods.

That correlation coefficients and effect sizes might require nonstandard and counterintuitive interpretations if large numbers of events are involved was forcefully argued in Abelson (1985), which highlighted the apparently small but nevertheless significant correlation between a professional baseball player’s skill and success at bat.¹⁴ Abelson pointed out that the correlation between batting skill and success with respect to any single at-bat for a highly skilled professional baseball player (batting average in the low .300s) as compared to a less skilled player (batting average in the low .200s) might appear quite small. In particular, Abelson calculated the correlation to be r = .056, where the corresponding effect size would thus explain less than 1/3rd of 1% of the variance between the batting outcomes of lower versus more highly skilled players (1985: 131). But the apparently small magnitude of the correlation and accompanying effect size, when considered over hundreds or thousands of at-bats, can nevertheless indicate important consequences, potentially accounting for the difference between baseball teams that win championships and teams with losing seasons.

With respect to the potential effects of implicit racial biases, Greenwald et al. (2015) illustrate via a hypothetical that even the small (by conventional standards) mean correlation between implicit measures and behaviors estimated by Oswald et al. (2013) (r = .14) might indicate a substantial impact at the group level. In particular, they raise the possibility that this correlation might implicate implicit racial biases in such significant discriminatory effects as an excess of nearly 10,000 police stops of racial minorities in New York City each year. Their hypothetical is informed by data but also invokes speculative assumptions, so it should not be confused with a descriptively accurate, causal explanation of New York City policing.¹⁵ Nonetheless, Greenwald et al.’s hypothetical makes a valid if narrow statistical point: the correlations and effect sizes at issue are not inherently indicative of trivial impacts. In their hypothetical, the potential impact of an r = .14 correlation between implicit racial biases and racially biased policing was calculated with data on nearly 200,000 police stops per year across 76 precincts, where New York City police “stopped an average of 38.2% … more of each precinct’s Black population than of its White population” (Greenwald et al. 2015: 558). Greenwald et al. (2015)’s hypothetical illustrates that given large variability, and a large number of events, even the lower end of mean correlations between implicit measures and behaviors estimated by Oswald et al. (2013), Carlsson and Agerström (2016) and Forscher et al. (2019) might indicate significant effects for society.

Greenwald et al.’s (2015) and Abelson’s (1985) examples imply that an alternative to Cohen’s guidelines for interpreting correlations and effect sizes in implicit bias research may often be appropriate. One such alternative framework for evaluating psychological research more generally is advocated by Funder and Ozer (2019: 11):

We offer … the following New Guidelines: … an effect with the size of r = .05 is “very small” for the explanation of single events but potentially consequential in the not-very long run, r = .10 is still “small” at the level of single events but potentially more ultimately consequential; an effect size of r = .20 is “medium” and of some use even in the short run and therefore even more important; and an effect size of r = .30 is “large” and potentially powerful in the short and long run.¹⁶

Funder and Ozer’s framework is likely more apposite than Cohen’s guidelines, in many circumstances, for interpreting the estimated mean correlations reported by competing meta-analyses of studies on implicit measures and behaviors. Arguably, this is the case for research on implicit racial biases in particular because they are potentially implicated, over relatively long timeframes, in a multitude of interactions among individuals perceived to be of different races.

2.5. Racially Biased Behaviors and Cumulative Effects

With respect to assessing the effects of implicit biases at the individual as opposed to group level, establishing that “small” correlations may nevertheless indicate significant effects for an individual often requires establishing that the relevant effects would accumulate if repeated over that individual’s life. That is, the relevance of implicit bias research for a single individual rather than a population is often determined by whether at least some of the behavioral effects found to be correlated with implicit measures build over time rather than dissipate (see Mallon & Kelly 2012 for discussion). Some kinds of effects do not accumulate. For example, the effects of perceptual and other kinds of stimuli can decrease for an individual despite repetition, as counter-processes of habituation facilitate environmental adaptation. Thus, an individual who moves to a large city can become accustomed to traffic noises rather than experience an accumulation of their initially disruptive effects. An important question for assessing research on implicit racial bias is whether we ought to view the effects of racially biased behaviors that have been correlated to implicit measures as more akin to traffic noises or batting outcomes.

Oswald et al. (2015) are skeptical that the racially biased behaviors studied by implicit bias researchers have real-world, cumulative effects. They remark, “[w]e do not doubt that the effects of small negative events can, in principle, accumulate over time with consequential effects,” but warn that “whether this happens for the outcomes studied in IAT research” should not be “simply stipulated” (2015: 568). Oswald et al. go further in their skepticism about the claim that “the small effects found in research laboratories translate into consequential real-world effects,” maintaining that such a claim “depend[s] crucially on untested and, arguably, untenable assumptions” (2015: 562).

It is reasonable for Oswald et al. to warn against simply stipulating that the behavioral effects studied in implicit bias research likely accumulate in real-world settings. But the assertion that the accumulation of such effects in real-world circumstances is “untenable” expresses an unwarranted level of skepticism. Implicit bias research has studied, including in field rather than lab settings, at least some behaviors whose effects would plausibly accumulate if repeated. For example, a field study by Rooth (2010) found that greater levels of negative implicit bias in Swedish employers against Arab/Muslim men positively correlated to fewer offers of interviews from those employers. While the correlation is “small” or “very small” by Cohen’s guidelines or a similar framework (r = .113), this type of effect would plausibly accumulate if repeated, with the capacity to alter the careers and life courses of the individuals affected. Oswald et al. (2015) acknowledge Rooth (2010)’s field study and its potential implications but counsel “caution in making broad claims for real-world meaning from this one study” (2015: 567).

Additional research not analyzed by Oswald et al. (2013) or Oswald et al. (2015), however, undermines their skepticism in two ways. First, additional field studies suggest the finding in Rooth (2010) is not a misleading outlier with respect to identifying plausibly real-world cumulative effects associated with implicit racial biases. Second, the most recent and extensive meta-analysis of studies on implicit measures and intergroup behaviors—Kurdi et al. (2018)—in general supports the external validity of implicit bias research, finding no differences among the analyzed studies as a “function of study setting” (2018: 15). On the first point, Cooper, et al. (2012) conducted a field study of doctors’ implicit cognitive stereotyping of patient race and found correlations to “visit length [and] speech speed” (2012: 981). That is, according to this study, doctors on average spend less time and speak faster with patients who are members of implicitly less favored races.¹⁷ Similarly, Hagiwaraet al.’s (2014) field study found that greater levels of anti-Black implicit bias among doctors predicted greater physician-patient talk ratios (i.e., the ratio of time physicians talk to the time patients talk) for Black patients (2014: 127). On the assumption that less time and lower-quality communication during doctor visits can lead to inferior health outcomes, both of these studies identify behavioral effects whose consequences, if repeated, would plausibly accumulate over an individual’s life. That is, repeatedly experiencing inferior mental or physical health outcomes would plausibly build over time into significant effects for someone who is affected compared to someone who is not. Thus, as with the impact of implicit racial biases at the group level, there is reason to believe the lower end of meta-analytic estimates of mean correlations between implicit measures and behaviors suggest implicit racial biases may have significant impacts at the individual level.

The upshot, with respect to the controversy over interpreting the magnitude of correlations and effect sizes as reported by competing meta-analyses of studies on implicit measures and behaviors, is that defenders of implicit bias research have a more compelling position. Even the smaller mean correlations between implicit measures and behaviors estimated by Oswald et al. (2013), Carlsson and Agerström (2016), and Forscher et al. (2019) are suggestive of potentially substantial discriminatory effects, as Greenwald et al. (2015) argues, “either because they can affect many people simultaneously or because they can affect single persons repeatedly” (2015: 553). This claim is not untenable in light of the best available evidence; it is plausible.

Despite the success of the above replies to critics of implicit bias research, the defense explored so far is only partial. Such replies do not establish that implicit measures have incremental predictive validity over explicit measures for predicting behavior. In other words, the above defense of implicit bias research has not shown that implicit measures add any predictive value to extant explicit measures. Nor does the defense address the worry that subsequent meta-analyses of studies on implicit measures and behavioral effects might follow a trend of finding ever smaller mean correlations and accompanying effect sizes.

3.1. Interpreting Kurdi et al. (2018)’s Findings

One might object that Kurdi et al.’s suggested reframing invites implicit bias researchers to cherry-pick a subset of variables associated with higher mean correlations between implicit measures and behaviors.²¹ Two points in response are worth considering. First, as I argued above, a mean correlation of r = .10 between implicit measures and intergroup behaviors, which Kurdi et al. find holds across all conditions, plausibly indicates significant effects at both the individual and group levels. Second, there are grounds for adopting Kurdi et al.’s tentative reframing. Among the most compelling is that a similar frame has been widespread for more than half a century in attitude psychology generally, where self-reports of explicit attitudes are widely understood to be better predictors of behaviors under some (theoretically expected) conditions than others (see Brownstein et al. 2020: 5–6, for discussion). In this light, Kurdi et al.’s suggested reframing with respect to research on implicit intergroup attitudes enjoys some plausibility. Ultimately, however, whether it is appropriate to emphasize conditions among Kurdi et al.’s findings that are associated with a higher estimated mean correlation between implicit measures and intergroup behaviors depends on both the nature of the variables that were found to impact Kurdi et al.’s estimate and those that were not.

With respect to the latter, Kurdi et al. report that ten variables they tested were not associated with a higher mean correlation than r = .10.²² While this might appear to lend credence to the objection that Kurdi et al.’s suggested reframing invites biased selection of variables, it is worth emphasizing that five of the variables found not to be associated with a higher mean correlation between implicit measures and intergroup behaviors would have undermined Kurdi et al.’s findings if they had been. That is, half of the variables that did not impact Kurdi et al.’s estimated mean correlation essentially tested for various forms of bias and invalidity in implicit bias research, including: publication status (published vs. unpublished), sample population (general, online, preselected, real-world, or student), sample origin (US vs. foreign), sample composition (only stigmatized participants, only non-stigmatized participants, or both), and study location (lab, online, or real-world) (Kurdi et al. 2018: 5). In the present context, this should reduce (but not eliminate) concerns related to biased selection of variables by Kurdi et al. (2018). Half of the variables that might support such a worry were in fact components of successful tests for flaws in implicit bias research, including publication bias, external invalidity, and various forms of selection bias.²³ Evaluating Kurdi et. al.’s suggested reframing also requires considering the variables that, according their findings, define conditions with a higher mean correlation than r = .10 between implicit measures and intergroup behaviors, including those involving race.

With respect to these moderating variables, the most informative interpretation of Kurdi et al.’s findings, for our purposes, highlights that variables they associate with a mean correlation of r = .23 between implicit measures and intergroup behaviors are plausibly associated with greater measurement accuracy and reflective of many attitudes and behaviors involving race in real-world settings. Additional variables associated with even higher mean correlations may enjoy a similar status, although their status is less clear. On this basis, as well as on the basis of general plausibility considerations on correlations and effect sizes in psychological research, I endorse an interpretation of Kurdi et al.’s findings as identifying one or more informative conditions under which mean correlations between implicit measures and many intergroup behaviors, including those involving race, are near r = .23 but likely below r = .37. Importantly, mean correlations between implicit measures and intergroup behaviors in this range imply significant behavioral effects according to all of the interpretational guidelines we have discussed, including Cohen’s.

Kurdi et al. identify a mean correlation of r = .23 between implicit measures and intergroup behaviors for studies satisfying the following two criteria (2018: 23):

(1) behaviors are measured using relative rather than absolute categories

(2) indirect tests for implicit associations use “high polarity” attributes as stimuli

(1) requires that behaviors be measured using more than two categories—for example, donating money, potentially of different amounts, to a White or Black student group as opposed to “absolute” categories, such as a “Yes” or “No” response (2018: 15). Given that the IAT and other implicit measures are themselves relative measures, measuring behaviors using relative rather than absolute categories plausibly allows for more precise calculations of correlations between the two. Further, behaviors appropriately characterized with more than two categories are also plausibly implicated in large numbers of interactions with the potential for discrimination in real-world settings, such as ranking multiple candidates for employment or admissions, as well as determining differently sized allocations of money, healthcare, time or other resources for individuals or groups perceived to be of different races. Thus, (1) is plausibly associated with both greater measurement accuracy and greater ecological validity.

(2) requires that “high polarity” attributes be used as stimuli in indirect tests (e.g., fat vs. thin) rather than attributes that are merely different but less opposed (e.g., sad vs. angry). Kurdi et al. speculate that “high-polarity attributes … may produce larger effects than low-polarity attributes … because they tap into a more cohesive network of mental representations” (2018: 15). Whatever the reason for a higher mean correlation between implicit measures and intergroup behaviors given (2), indirect tests such as the race IAT often use attributes whose differences exhibit high polarity when measuring negative and positive implicit associations to racial groups. Their results suggest the existence in many individuals of implicit racial attitudes with associated attributes that exhibit high polarity. This in turn supports a correlation of r = .23 (and perhaps higher) as informatively characterizing relations between many implicit racial biases and racially biased behaviors.

Kurdi et al. identify three additional criteria such that, if the studies analyzed satisfy them in addition to (1) and (2), the mean correlation between implicit measures and intergroup behaviors rises to r = .37 (2018: 23). But the relevance of these additional criteria to measurement accuracy or external validity is less clear than for (1) and (2). Kurdi et al.’s first additional criterion requires a standard IAT be used rather than an alternate indirect test. It may be that the IAT is a superior measurement tool to other indirect measures, perhaps because of its greater internal consistency (cf. Bar-Anan & Nosek 2014: 675). On the other hand, it is unclear why variants designed to improve on the IAT would fail to do so. Thus, the import of this criterion is more suggestive than probative.

Kurdi et al.’s second additional criterion requires a “high correspondence” between implicit attitudes and behaviors. This criterion is intended to be sensitive to Ajzen and Fishbein’s (1977) principle of correspondence, according to which attitudes are better predictors of behavior when there is clear correspondence between the content of an attitude and the behavior being measured. This criterion may also be associated with many relations between implicit racial attitudes and behaviors outside an experimental setting. However, Kurdi et al. report blind coding of this variable disagreed significantly with non-blind coding (2018: 21). So again, caution is warranted in assessing the import of this second criterion. A final criterion requires that studies have a “declared focus” on identifying correlations between implicit measures and intergroup behaviors. While this may be of interest as a methodological moderator for a meta-analysis, it does not directly bear on greater measurement accuracy or external validity. Thus, despite a mean correlation of r = .37 featuring in Kurdi et al.’s abstract (and in subsequent references to their meta-analysis, e.g., Brownstein et al. 2019: 9), the grounds are tenuous for regarding a correlation of this magnitude as informatively characterizing relations between measures of implicit associations and intergroup behaviors, including those involving race.

Instead, there is some reason to be skeptical given that a mean correlation of r = .37 between implicit measures and intergroup behaviors might render implicit bias research implausibly successful. As Funder and Ozer comment, in light of the general complexity of relations between cognition and behavior, as well as the history of psychological research:

A “very large” effect size (r = .40 or greater) in the context of psychological research is, we suggest, likely to be a gross overestimate that will rarely be found in a large sample or in a replication. Smaller effect sizes are not merely worth taking seriously. They are also more believable. (2019: 11)

In this light, mean correlations between implicit measures and intergroup behaviors near r = .23, and perhaps ranging modestly higher, are more plausible than a mean correlation of r = .37.²⁴ Nevertheless, according to the interpretation of Kurdi et al. (2018) I advocate, this meta-analysis has identified one of the highest ranges to date of mean correlations between implicit measures and intergroup behaviors, including those involving race. Moreover, according to reasonable interpretational frameworks advocated by Funder and Ozer (2019) and Gignac and Szodorai (2016), these correlations and accompanying effect sizes are in the “medium” to “large” range for human behavioral research, and suggest implicit social biases, including implicit racial biases, correspond to significant behavioral effects at both the individual and group levels.

Kurdi et al.’s (2018) findings support implicit bias research in other important ways, including defending it from worries related to publication bias and external validity. More specifically, Kurdi et al. (2018: 10) found that the unpublished materials they reviewed generally showed slightly larger effects than published results. They also tested for greater model fit under the assumption that unpublished results showed smaller effect sizes than published results, yet found no such fit (2018: 10).²⁵ On these bases, Kurdi et al. conclude: “unlike other fields of psychology, the study of implicit–criterion relationships is unlikely to be plagued by a widespread file drawer problem” (2018: 10). In this way, Kurdi et al. (2018) at least partially defend their findings from the general worry that meta-analyses often merely aggregate the results of biased studies (Romero 2016). With respect to external validity, Kurdi et al. analyzed whether correlations between implicit measures and behaviors were affected when tested for “under the less controlled conditions of online and field studies” (2018: 15). And their meta-analysis found “no difference” in average correlations between implicit measures and intergroup behaviors “as a function of study settings” (2018: 15). Thus, as with publication bias, their findings defend implicit bias research from the worry that correlations between implicit measures and intergroup behaviors, including those involving race and ethnicity, hold solely in experimental as opposed to real-world settings. In the next subsection, I situate these findings within the context of competing meta-analyses of implicit bias research and the debate over whether implicit racial biases likely have significant discriminatory effects.

3.2. Taking Stock of Competing Meta-Analyses of Studies on Implicit Measures and Behaviors

Let us take stock of the trajectory of meta-analyses of studies on implicit measures and behaviors over the past decade. Despite a mid-decade trend in which meta-analytic estimates of mean correlations between implicit measures and behaviors were diminishing, in fact meta-analyses during this time did not report mean correlations or accompanying effect sizes that would be insignificant for large populations in the short term or for individuals over longer time frames. Now, the most extensive meta-analysis to date has estimated greater mean correlations under conditions plausibly reflective of many real-life circumstances, and where measurement of implicit biases and intergroup behaviors, including those involving race, may have higher degrees of accuracy. Further, this meta-analysis has portrayed implicit measures as adding significant predictive power to explicit measures and defended research on implicit bias from criticisms relating to publication bias and external validity.

While the positive view of implicit bias research just sketched is warranted in light of the best available evidence, a cautious stance is nevertheless appropriate given that research on implicit social cognition is an active, unsettled domain of inquiry. Kurdi et al. themselves highlight systematic inadequacies in implicit bias research, including inattention to measurement error and problematic levels of internal consistency (2018: 16). Moreover, their meta-analysis relies on methodological inclusion criteria for behavioral effects that are sensitive to author prediction. These inclusion criteria are reasonable, but not necessarily superior to alternatives associated with smaller estimated mean correlations between implicit measures and behaviors.

Nevertheless, Kurdi et al. (2018) provides an important defense of research linking implicit measures to intergroup behaviors, including those involving race. While their meta-analysis should not be taken as the last word on correlations between the two, nor on the uniquely predictive powers of implicit measures of bias, neither should its substantial evidential force be dismissed. Kurdi et al.’s findings clearly suggest that measures of implicit racial bias significantly correspond to racially biased behaviors. In broad terms, the best available evidence from social psychology supports the conclusion that implicit racial biases have discriminatory effects that are significant at both the individual and group levels. However, the research on which this conclusion is based primarily investigates questions of correlation rather than causation. Consequentially, I will consider the role research on implicit racial bias might play in accounts of racial discrimination and inequality both on the assumption that implicit racial biases causally influence racially biased behaviors and on the assumption they do not.

4.1. A Causal Interpretation of Research on Implicit Racial Bias

According to a “causal interpretation” of research on implicit racial bias, findings on correlations between implicit racial biases and racially biased behaviors track, in some cases, the causal influences of the former. For example, a causal interpretation of Kurdi et al.’s (2018) finding that implicit measures of bias have incremental predictive validity compared to explicit measures implies that, in some cases, implicit biases make unique causal contributions to biased behaviors. Thus, on a causal construal, implicit bias research suggests implicit racial biases are significant causal antecedents of at least some racially biased behaviors. Plausibly, understanding the causes and effects of racially biased behaviors can further our understanding racial inequality. In which case, research on implicit racial bias, on a causal interpretation, provides reason to anticipate that it will contribute to such an understanding.

In addition, a causal construal of implicit racial biases suggests the study of their manipulation and mitigation may play a useful role in efforts to reduce discrimination and ameliorate inequality. I am not here arguing that research on implicit racial bias could provide insight into racial discrimination and inequality equal to or greater than alternative accounts, such as those focused on explicit racial biases or non-psychological, structural causes. Rather, I maintain it is plausible that research on implicit racial biases, if a causal interpretation is apposite, will play a role, likely alongside other kinds of research, in our best accounts of racial inequity and efforts to address it.

4.2. A Non-Causal Interpretation of Research on Implicit Racial Bias

On a non-causal interpretation of implicit racial bias research, implicit racial biases are significantly correlated but not causally related to racially biased behaviors, perhaps because both are products of common social-environmental causes. Compared to a causal interpretation of implicit bias research, a non-causal interpretation casts implicit bias research in a less important light. For example, if implicit racial biases are construed non-causally, research on their short-term manipulation (Lai et al. 2014; Lai et al. 2016) would likely be of limited relevance to accounts of racial inequality or efforts to address it. Nonetheless, I believe we may expect, even on a non-causal interpretation, that implicit bias research will play a useful if limited contributory role.

A central reason for such an expectation is that variables that are correlated but not causally related to significant effects are nevertheless often used to predict them. As Forscher et al. point out, in their meta-analysis that many interpret as supporting a non-causal construal of implicit attitudes, demographic variables such as life expectancy are often used “to predict consequential outcomes within a population, despite lacking causal force themselves” (2019: 544). Forscher et al. anticipate in particular that even if implicit biases are “causally inert … implicit measures could be used to predict the prevalence of … behaviors within a population” (2019: 544). Such a use for implicit measures is plausible in light of Kurdi et al. (2018)’s finding that implicit measures have, in general, incremental predictive validity compared to explicit measures for predicting intergroup behaviors. As applied to implicit measures of racial bias, their finding implies that these measures, in some circumstances, better predict racially biased behaviors than explicit measures.

Hehman et al.’s (2017: 395) findings illustrate predictive power of this kind. Their study found that implicit but not explicit measures of White prejudice and stereotype in a region predict disproportional use of lethal force by police on Blacks in that region, relative to regional base rates of Blacks in the population. On a non-causal construal of implicit racial biases, which Hehman et al. consider, implicit measures of White racial prejudice and stereotyping in a region may nevertheless register useful information about the actual causes of law enforcement’s disproportionate lethal force on Blacks (2017: 398). Plausibly, understanding racial inequality can be furthered by understanding the causes and effects of such racially biased behaviors as law enforcement’s racially disproportionate use of lethal force. Thus, there appears little reason to eschew tools with unique powers for predicting such behaviors. Instead, even if implicit racial biases are construed non-causally, the current state of implicit bias research suggests implicit measures of racial bias may well play a substantive role in our best explanatory accounts of racial discrimination and inequality.