Legal Commentary

Spreadsheets vs. Mean Streets

By Jack M. Balkin

Last Monday, the A.C.L.U. of Southern California released a report that Jonathan Borowsky and I wrote that analyzes more than 700,000 pedestrian and motor-vehicle stops conducted by the Los Angeles Police Department over a 12-month period from July 2003 to June 2004.

As I said in a Los Angeles Times op-ed:

We found persistent and statistically significant racial disparities in policing that raise grave concerns that African-Americans and Latinos in Los Angeles are, as we put it in the report, “over-stopped, over-frisked, over-searched, and over-arrested.” After controlling for violent crime rates and property crime rates in specific neighborhoods, as well as a host of other variables, we found the following:

For every 10,000 residents, about 3,400 more black people are stopped than whites, and 360 more Latinos are stopped than whites. Stopped blacks are 127 percent more likely to be frisked, and stopped Latinos are 43 percent more likely to be frisked than stopped whites.

Stopped blacks are 76 percent more likely to be searched, and stopped Latinos are 16 percent more likely to be searched than stopped whites.

Stopped blacks are 29 percent more likely to be arrested, and stopped Latinos are 32 percent more likely to be arrested than stopped whites.

The evidence regarding over-stopping, over-frisking, and over-searching is particularly compelling. Although stopped blacks were 127 percent more likely to be frisked than stopped whites, they were 42.3 percent less likely to be found with a weapon after they were frisked, 25 percent less likely to be found with drugs, and 33 percent less likely to be found with other contraband.

Frisks and searches of blacks and Latinos were systematically less productive than those of whites:

Our report has helped unify labor and management — as both the union president and the Chief of Police quickly rejected the findings of the study. Union President Tim Sands dismissed our report as “an exercise that might work on a spreadsheet at Yale, but doesn’t work on the streets of Los Angeles.”

This is an elegant (anti-elite) turn of phrase; but they’re fighting words to people like Steve Levitt, who have used statistical packages at a distance to uncover unexpected truths about our criminal justice system. I respond to some of their particular criticisms in my op-ed:

Police Chief William J. Bratton quickly rejected these findings, primarily because the study used data that were more than four years old. This is a fair point. But we had no other choice: The department has not released the more recent stop data that it has been collecting, nor has it analyzed the more recent data to test for racial disparities. If Bratton is truly confident that unjustified racial disparities are a thing of the past, he should be able to show the change in the current data. I would be happy to organize a group of respected academics to help analyze it.

Bratton also asserted that the report was flawed because we failed to control for the race of both officers involved in the stop. On this point, Bratton is simply wrong about how to conduct a statistical analysis. When testing for unjustified racial disparities in who is stopped by the police in cars and on the street, it’s inappropriate to control for the race of either of the officers. The likelihood of being stopped, frisked, or arrested shouldn’t turn on whether a black, Latino, or white officer was involved.

As an ancillary test (after we’d calculated the general disparities), we did look at the officers involved, and we found that the racial disparities in the likelihood of arrest were substantially lower when at least one of the stopping officers was the same race as the suspect.

For example, we found that the black arrest disparity was 9 percentage points lower when at least one of the stopping officers was black. Bratton should be troubled that there is less disparity when the officer is the same race as the person stopped, as that result adds credibility to the idea that the disparities in different-race interactions may be because of racial bias.

The president of the Los Angeles Police Protective League, Tim Sands, even more harshly rejected the results of our report. Sands said I appeared to start with my conclusions and then “worked data to fit.” This is a vague charge, but one way to respond to the concern is with transparency. I’ve posted the data I used in the report and the associated statistical files to the internet so that other academics can easily double-check the report’s analysis.

Sands has argued that the results are not valid because officers often don’t know the race of the suspect when they decide to pull over a car. That may or may not be true. But our study looked not just at motor vehicle stops, but at pedestrian stops as well, which also showed racial disparities. We also found that, once people were stopped, officers were more likely to frisk, search, or arrest African-Americans and Latinos than whites. At the point of making these decisions, officers can certainly see the apparent race of the suspects.

It is particularly telling that neither Bratton nor Sands responded to the evidence that the frisks and searches of minorities systematically produced less evidence of crime than the frisks and searches of whites. It is implausible that higher frisk and search rates are justified by higher minority criminality, when these frisks and searches are substantially less likely to uncover weapons, drugs, or other types of contraband. Independent of racial disparity, it is a sign of ineffective policing to have officers engage in such a large number of fruitless searches.

Sands charges that I cannot use data to “prove what 9,700 individual officers are thinking when they make traffic stops.” But if he thinks that is what I tried to do, he seriously misreads the report. I never suggested that the data show what an officer might be thinking, and I was careful not to attribute the disparities to conscious discrimination on the part of individual officers.

What the report finds is that there are statistically significant racial disparities in a variety of police behaviors that are not explained by legitimate police concerns, such as the local crime rate — or, in the cases of frisks and searches, the likelihood of actually uncovering contraband.

My inability to probe the minds of officers does not make my results less important. The report shows that people of color in Los Angeles experience harsher treatment by police that doesn’t appear to be justified by any legitimate law-enforcement concerns. The L.A.P.D. can’t just deny that racism is involved and let the matter rest; it should take steps to address that inequality.

So what steps should be taken? My report makes a series of recommendations. But one of the simplest is my recommendation that:

[T]he department should require that all existing and new officers take the Implicit Association Test (IAT) and retain the results of these tests. The IAT is a free internet test which takes about 10 to 15 minutes to complete. It produces a measure of unconscious bias with regard to the attribute being tested. For example, the black/white IAT produces a measure of whether an individual has unconscious negative associations with photographs of African-Americans relative to photographs of whites.

Collecting information about the IAT [results] of individual officers would provide a powerful validation test of the estimated racial disparities reported in this report. If the individual officer disparities estimated [in the report] are positively correlated with the disparities estimated by the IAT, it would be very hard to accept that the policing disparities were justified by suspect or stop attributes.

The willingness of the department and the union to collect data on the possibility of individual officer bias will itself be a test of how serious they really are about responding to the issue.

By the way, the IAT is not just for the L.A.P.D.; anyone can take the test for free at Harvard?s IAT site. There is something powerful about learning whether you can keep yourself from treating photographs of African-Americans differently than photographs of European-Americans.

Full post as published by Balkinization on October 28, 2008 (boomark / email).