In Search of an Objective Risk Continuity Assessment: Developing an Objective Assessment for Police Deadly Force Encounters

Study Goals and Objectives

The goal of this study was to identify a means of increasing compliance with both Tennessee v. Garner and Graham v. Connor. Specifically, the study aimed to develop a method of empirically examining the continuity of deadly force claims in police uses of deadly force while considering objective situational factors that might have contributed to the decision to use deadly force. It should be clear that this study is not designed to develop a punitive tool with which to further burden law enforcement agencies but to build an instrument that can help address compliance with regulations governing use of force. A method of examining the level of objective support for the necessity of deadly force can be a valuable tool to support law enforcement agencies as they continue to undertake an increasingly dangerous job. Law enforcement executives could benefit greatly from a tool that could empirically assess the continuity of situational threats reported by their officers. Such an empirical assessment tool could also be used to further focus and enhance law enforcement training in the area of deadly force applications.

The result of the study is the Objective Risk Continuity Assessment (ORCA). The ORCA instrument uses a weighted Guttman index and produces a range from 0 (no risk of injury to an officer) to 10 (the greatest risk of injury to an officer). The ORCA index was designed using concepts found in the literature on police use of deadly force.

Literature Review

Many criminologists have set out to determine the factors associated with police decisions to use force. Studies have determined that there are several different factors and theories that influence and potentially explain the decision to use force against suspects (e.g., Alpert & Smith, 1994; White, 2002). Although this study was not designed to replicate previous analyses, the researchers chose to structure the new ORCA index in a similar way.

Some studies (Garner, Maxwell, & Heraux, 2002; White, 2002) suggested that working environment conditions play a prominent role in shootings involving unarmed suspects. Studies have consistently determined that the police subculture and normative expectations of officers may influence an officer’s perception of what is considered reasonable in use of force situations (Alpert & Smith, 1994; Maskaly & Donner, 2015; White, 2002). Maskaly and Donner (2015) integrated social learning and terror management theory in their study to explain officer shootings of unarmed suspects. Because situational variables have consistently been discussed in use of force studies (White, 2002), the ORCA index was constructed on the basis of observable situational factors that logically should inform an officer’s decision to use deadly force.

Regardless of any tangential correlates, all law enforcement deadly force is governed by two case law decisions. Current standards for evaluating police use of force issues were established by Supreme Court decisions in Tennessee v. Garner (1985) and Graham v. Conner (1989). In 1985, the Supreme Court ruling in the Tennessee v. Garner case established that the use of deadly force against a fleeing suspect who is unarmed or poses no threat of danger is a violation of the suspect’s rights under the Fourth Amendment. The court found that use of deadly force in such situations constitutes an illegal seizure of the person. This principle was extended to include not only deadly force incidents but all excessive force cases in which the person is a “free” citizen (Ross, 2002). Later, additional criteria for evaluating police use of force incidents were established by the Supreme Court decision in Graham v. Conner.

In 1989, the Supreme Court decision in Graham v. Conner established that excessive force claims arising from arrests, investigatory stops, or other seizures must be analyzed under the Fourth Amendment’s “objective reasonableness standard.” Although there is no precisely established application of the objective reasonableness standard, the court established criteria that should be considered when evaluating an officer’s decision to use force. These criteria include severity of the crime, whether the suspect poses an immediate threat to officers or others, and whether the suspect is actively resisting arrest or attempting to evade arrest (Hough & Tatum, 2010).

Methodology

This study used a quantitative cross-sectional design. The data were collected from open source reports in the Washington Post on deadly force encounters (N = 300) by law enforcement in the United States from 2014 to 2018. A content analysis of the open source data was used to identify case factors for the study. Case information was coded using a manifest coding procedure. The data were collected and coded by two researchers. The inter-rater reliability of the researchers, tested using an interclass correlation coefficient, was found to exceed the acceptable standard of .700 (rIC = .809, p = .012). The cases were coded using a proprietary index of factors that may increase risk to an officer in a confrontational setting.

The researchers used the following ten situational risk components in the scale:

1. Multiple suspects were present

2. Suspect was menacing the officer

3. Suspect had a weapon

4. Incident occurred in an isolated area

5. Officer was alone at the scene

6. Officer’s weapon was lost

7. Officer had lost communications device

8. Officer had lost (did not have) chemical munitions/pepper spray

9. Officer had lost (did not have) taser

10. Officer had lost (did not have) baton

This produced a scale from 0 (no objective risk to the officer) to 10 (the highest objective risk to the officer).

The researchers realized that, although situational risk may be identifiable, it is nonlinear. Some of the situational risk components create an exponentially higher risk to officers than others. Risk factors such as item 3, suspect possessing a weapon, were weighted more heavily (x2) in the overall scale computation to reflect the exponential increase in risk presented to the officer. Additionally, great care was taken to code the data in a more conservative manner to avoid creating a perception that unfairly diminished the risk faced by law enforcement officers. Typically more conservative coding (coding that favors the use of force by the law enforcement officer) was used with “sticky cases.”

Throughout the data collection process, there were individual cases that defied simple coding. These sticky cases were examined more thoroughly and subjected to both manifest coding and latent analysis. They were examined and coded based on the objective probability of risk to the officer rather than the objective possibility of risk to the officer. For example, one case contained data on a suspect who was killed by law enforcement officers because the suspect was seen holding a gun. Eventually it was determined that the gun was unloaded at the time and could not have harmed the officer. This case was coded as an objective risk because the suspect had a weapon. The fact that the weapon was unloaded was irrelevant to the coding process; the suspect’s weapon still presented a possible risk to the officer. In cases where situational risk components could not be verified through either manifest or latent analysis, the researchers treated those factors as missing data.

Study Findings

Analysis of the sample data indicated that the suspects killed in deadly force situations were mostly male (89.7%, n = 269). African Americans comprised the largest proportion of suspects killed (53.7%, n = 146) and Caucasians the second largest group killed (29%, n = 79). Minorities as a whole comprised the vast majority of suspects killed in deadly force encounters with the police (71%, n = 193). One concern that is often raised in deadly force discussions is mental illness. The analysis found that the vast majority of suspects (93.6%, n = 270; missing = 9.7%, n = 29) killed in deadly force situations were not reported to be mentally ill (see table 1).

The average age of suspects killed in deadly force situations was 32.25 years old (median age = 30). Although the age of the suspects was not skewed, there was a high degree of heterogeneity within the sample (SD = 12.410). Approximately half of the suspects in the sample (49.3%, n = 148) were under the age of thirty; 74.3 percent (n = 223) were aged forty or younger.

On a 10-point scale measuring objective situational risk to officers in deadly force situations, the average objective risk level was relatively low (INSERT IMAGES = 3.21, median = 4). The objective risk levels found in the sample were also very similar, as evidenced by a relatively low degree of heterogeneity (SD = 1.58). The vast majority of the deadly force situations in the sample (80.1%, n = 85) had an objective risk of 4 or less on a 10-point scale, and almost 10 percent (9.3%, n = 22) had an objective risk of 0. It is important to note that no case in the sample had a maximum risk score of 10. Because information pertinent to this variable was not found in the open source documentation for 64 of the 300 cases (see table 1), 21.3 percent of the sample had missing data.

Next the researchers examined the binary associations between objective risk and the study variables. In addition to the descriptive statistics, the researchers also examined the individual components of the ORCA scale in order to better understand the prevalence of each situational threat. By far, the most represented situational threats to officer safety in the study were the possession of a weapon (62%, n = 186) and the suspect menacing the officer (77.3%, n = 231). These two situational variables are mutually exclusive. It is possible to be in possession of a weapon and not actively menacing an officer. Therefore, the researchers computed a composite measure to identify the cases in which the suspects were both in possession of a weapon and actively menacing the officer. In approximately half of the cases (53.5%, n = 160), suspects were both in possession of a weapon and menacing officers.

Less than one-quarter of the cases (16%, n = 48) involved multiple suspects, and only a very small number of cases (2%, n = 6) occurred in an isolated area. Additionally, officers were alone on the scene in only 18.7 percent (n = 56) of the cases. The officers almost never lost their weapons (0.3%, n = 1), their chemical munitions/pepper spray (0%, n = 0; 16.7% missing data), or their taser (1.3%, n = 4), and they never lost their communications devices (0%, n = 0). These data suggest that, despite other situational threats, the officers appeared to retain their protective equipment (see table 2).

A bivariate analysis of the data found only one statistically significant relationship, that between a suspect having minority status and being a youthful offender (rtet = .173, p = .004). The variable age was dichotomized into a binary variable called youthful offender (age 0–25 = 1 and age 26+ = 0) whereas the ethnicity variable was dichotomized into the binary variable minority status (1 = minority and 0 = nonminority). The dual dichotomization produced a tetrachoric correlation that was moderately weak. The weakness of this relationship was to be expected due to dual attenuation produced by two binary variables used in a Pearson product-moment correlation matrix. This finding suggested that having minority status was statistically related to being a youthful offender. Despite this being the only statistically significant relationship in the bivariate analysis, there are several other relationships that warrant discussion, precisely because they were not statistically significant.

There was no statistically significant relationship between the objective risk score and minority status (rpb = -.0743, p =.275), suspect sex (rpb = .020, p =.761), youthful offender status (rpb = .064, p = .328), or mental illness (rpb = -.053, p = .427). These findings suggest that the objective risk to officers in deadly force situations was not statistically related to a suspect’s minority status, youthful offender status, sex, or mental illness. Essentially, the objective risks to officers in deadly force incidents were stable across all types of encounters in the data.

As a method of cross-validating these findings, the researchers conducted a separate means analysis on the data (see table 3). The logic for conducting the means analysis was that, despite there being no statistically significant relationships between the ORCA risk score and the study variables, the risk may vary, in an appreciable way, among the categories of the study variables. As shown in table 4, the means analysis found that there was no statistically significant difference in the objective risk between minority and nonminority suspects (p = .275), between males and females (p = .761), or between youthful offenders and non-youthful offenders (p = .328) nor was there any difference in objective risk between suspects with and without mental illness (p = .427).

Finally, the researchers conducted a multivariate analysis to determine whether any of the study variables might predict ORCA risk scores. This analysis found that the overall ordinary least squares (OLS) regression model was not statistically significant (F = .681, p = .565). Even if the model had been significant, the data suggested that it would have predicted less that 1 percent of the variation in objective risk in deadly force situations (R2 = .009). Additionally, the model showed that the objective risk could not be predicted by minority status (b = -.073, p = .284), sex (b = .054, p = .428), or age (b = .034, p = .622) (see table 4). These findings raise several interesting questions that merit further discussion.

Study Discussion

The first finding that warrants discussion stems from the actual ORCA scores. Although all of the cases selected for this study involved the use of deadly force, the average level of objective risk was relatively low (INSERT IMAGE = 3.21, median = 4). Logically, one might expect the objective risk to be much higher when deadly force is deployed. Additional findings show that a large majority of deadly force situations in the sample (80.1%, n = 85) had an objective risk of 4 or less on a 10-point scale. These findings call into question the factors used to assess the need to deploy deadly force.

The ORCA score components that produced the most common threats to officer safety in the study were the possession of a weapon (62%, n = 186) and the suspect menacing the officer (77.3%, n = 231). The analysis showed that suspects who were in possession of a weapon while menacing an officer represented about half of the suspects in the study (53.5%, n = 160). Of course it is not clear exactly how the suspects were menacing the officer. The conceptualization of menacing behavior can be very subjective depending on the situation.

Next there are issues surrounding the demographic findings. The data showed that the number of African Americans suspects killed (53.7%, n = 146) was disproportionate to their representation in the United States. Although Caucasians comprised the second largest group of suspects killed (29%, n = 79), their numbers were disproportionately underrepresented. Minorities as a whole (African Americans, Latinx, and Asians) comprised the vast majority of suspects killed in deadly force encounters with the police (71%, n = 193). With regard to the low ORCA score, these findings are troubling beyond the simple disproportionate representation in society. These data suggest that minority suspects were killed more frequently but under circumstances that had objectively low risk for law enforcement officers.

Taken as a whole, the ORCA scores do not seem to support the use of force in all of the cases in the sample. The presence of additional tools and technologies did not seem to have much bearing on the decision as to whether to use deadly force. When confronting an individual with a bladed weapon, a law enforcement officer has the legal right to deploy a firearm to stop the suspect. However, is it possible to stop the armed suspect equally well with a less-than-lethal option such as a taser or a baton? Must officers always select the exact same level of force as the suspect, or can a lower level of force be used to affect the same outcome? Ultimately, we face the question of how much risk is enough risk to take an individual’s life. Perhaps this is the fundamental question at hand, a question that may raise more concerns in the answering than in the posing.

This study suffered from several limitations that also bear discussion.

Study Limitations

It is important to acknowledge that one of the authors has served as a sworn law enforcement officer. Although no analysis is totally unbiased, the research team took multiple steps to identify and mitigate any unconscious law enforcement bias. Secondary and tertiary vetting was used as an unconscious bias check. The result is an academic inquiry that is as fair and unbiased as possible.

An important study limitation is the sample size. Although 300 cases provided adequate statistical power for the study analyses, a larger sample may have provided greater ability to probe the data. Having a larger sample size may have also been beneficial for using additional statistical techniques. Future studies should endeavor to expand the sample size to include not only more cases but longer spans of time. Having cases from various time periods provides a means to determine if there are any meaningful variations that correlate with sociocultural experiences.

Another limitation concerns missing data. Some case-related facts were not readily available in the publicly available reports. These facts were necessarily coded as missing data. Although there are a number of methods available for addressing missing data (e.g., means replacement or log odds ratios), the researchers chose to analyze the data without using any of these techniques. By choosing not to address the missing data with mean replacement, the researchers precluded any argument that the findings were in any way controlled or produced through data manipulation. The ORCA score variable contained the largest amount (21.3%, n = 64) of missing data. Future tests of the ORCA scale should address any missing data in archival sources. Future analyses of agency data should be far less susceptible to issues of missing data due to relaxed restrictions on data access. There may also be a limitation with the scale construction itself.

The loss or absence of chemical munitions (e.g., oleoresin capsicum spray) or tasers creates an artificially inflated risk score if the agency does not provide these tools. For the purpose of this study, the researchers coded data more conservatively (in favor of law enforcement) so that this was less of an issue. However, in future studies it may be necessary to develop a different method for addressing law enforcement agencies that do not issue less-than-lethal enforcement tools.

The final study limitation deals with assessing the perception of risk. In this study the researchers attempted to assess only the risk to law enforcement officers. Because the most consistent claim made by officers in defense of the use of deadly force was imminent danger to their own lives, the researchers did not assess risk to a third party. It is possible that the officers in the study cases used deadly force due to their assessment of risk to a third party. If this was the case, the ORCA scale has no component that addresses this concern. Future studies should include a weighted component that accounts for the possible risk to a third party.

Conclusions

Law enforcement executives can use the ORCA as part of the formal review process in deadly force incidents to help gauge the decision-making processes of the officers involved. If officers are making decisions to use deadly force in ways that are inconsistent with policy, an ORCA analysis will highlight the incongruence. Through early diagnosis of this faulty decision-making process, law enforcement executives can implement interventions to address the problem. Additional academy training, prolonged field training, or increased supervision may help to correct decision-making processes that are not conforming to departmental policy requirements.

Although it is vital to correct any problems associated with the decision to use deadly force, the ORCA tool has other benefits. For example, it may help to diagnose gaps in an agency’s equipment allocations. If an officer uses deadly force because of a lack of alternative options, an ORCA analysis would point out this problem. In agencies that do not issue chemical munitions or electronic munitions, officers may be more likely to select deadly force to control a situation simply due to a lack of other options. An ORCA analysis may also help identify officers who have been given proper training and equipment but who simply lack the temperament to make less restrictive choices. Finally, using the ORCA tool allows law enforcement executives to remain in compliance with the Supreme Court ruling in Graham v. Connor.

© 2020 International Consortium for Social Development