An all-killer, no-filler roundup of recent research on cities and criminal justice
It has been an exciting few months for those of us who like to stand at the intersection of justice, research and policy and gawk. In addition to the furor caused by Megan Stevenson’s essay “Cause, Effect, and the Structure of the Social World,” (covered extensively elsewhere in Vital City), there was also a dust-up between Jennifer Doleac of Arnold Ventures and Nick Turner of the Vera Institute of Justice.
It all began, as many inside-baseball micro-controversies do these days, on the platform formerly known as Twitter. In late November 2023, Doleac sent out a series of tweets announcing, “Another day, another highly-touted matched comparison group study with completely implausible and misleading ‘causal’ findings.” The target of her ire was a recent study by Vera about the impact of college-in-prison programs in New York. Doleac’s verdict was dismissive: The report was “research malpractice” that provided “no useful information about whether this NY program is having beneficial effects.”
A few weeks later, Turner decided to return fire. He sent out a missive to Vera’s entire email list defending the study’s methodology. According to Turner, Doleac was guilty of overemphasizing randomized controlled trials. “By privileging a rigid research standard and creating divisions over methodological differences, Doleac is giving comfort to the opponents of reform,” Turner declared.
We are not inclined to adjudicate an argument between two prominent friends of Vital City. But, at the risk of being Pollyanna, we will say that the controversy sparked a number of interesting conversations, both online and IRL, about the purpose of social science research. So, in an odd way, we are grateful for the dispute.
This edition of “What We’re Reading” tries to pick up on some of the themes of the Doleac-Turner disagreement, including the value of randomized trials, the potential role played by selection effects and the difficulty of translating evidence into action. The usual rules apply: We are nonacademics reviewing academic literature. We aren’t checking the math, just attempting to distill the gist for the benefit of our readers. As always, we hope these summaries will encourage you to check out the source material for more detail.
Bottlenecks for Evidence Adoption
Stefano DellaVigna, Woojin Kim and Elizabeth Linos
This essay for the Journal of Political Economy asks a fundamental question: How often are the innovations successfully tested in randomized controlled trials actually implemented over the long haul? (Spoiler alert: not as often as one might hope.)
In 2015, the U.K.-based Behavioural Insights Team opened a North American office. Like other so-called “nudge units,” the office sought to use behavioral science to improve the delivery and impact of government services. The office worked with dozens of American cities to launch randomized controlled trials involving low-touch/low-cost “nudges” — for example, sending out letters to encourage people to pay their utility bills on time.
The research team went back and looked at 73 randomized controlled trials instigated by the Behavioural Insights Team. What percentage of the nudges ended up being adopted after the trial period was over? The answer: 27%. (Remember that these were small communications tweaks, not complicated or expensive interventions.)
The strength of the evidence made little difference to adoption after the trial. That is, the small nudges were adopted at about the same rate regardless of the outcome of the randomized trial — the experiments with negative findings were adopted 25% of the time and those with statistically significant, positive findings were adopted 30% of the time. The strongest predictor of adoption was whether the communication in the trial was pre-existing or not; in the 21 trials in which the nudge was inserted into a pre-existing communication, the adoption rate was 67% (14 out of 21). This leads the authors to argue that organizational inertia is an important factor to reckon with in advancing evidence-based policies. The pull of the status quo ante is powerful indeed.
The Bottom Line: “The bottlenecks [to the adoption of evidence] are substantial: The innovations from the RCTs yield only about one-third of their potential direct benefits. This is because the rate of adoption is fairly low, 27%, and is only modestly sensitive to the effectiveness of the intervention. As a consequence, several high-return nudge innovations are not adopted by the city in years subsequent to the experiment. … To an extent, this is bad news for evidence-based policy-making. But there is good news too: the barriers to adoption, in our context, do not appear to be due to intractable problems such as political divisions or funding challenges for the roll-out, but more ‘simply’ due to organizational inertia.”
David M. Hureau, Anthony A. Braga, Tracey Lloyd and Christopher Winship
This article for Criminology performs a valuable public service, analyzing the existing research literature about Chicago’s Cure Violence and other street outreach programs. As we have previously detailed in these pages, streetwork programs, because they offer a visible response to gun violence that does not prioritize law enforcement, have become enormously popular over the past decade.
But even as Cure Violence and similar initiatives have proliferated across the country, questions have been raised about their effectiveness. According to Hureau and his co-authors, who review evaluations of anti-gang streetwork dating back to the 1950s, these programs “may have been oversold.” They lament that “the recent research literature into the efficacy of streetwork has offered evidence that such interventions are effective, generate mixed results, or can even be harmful.”
Picking up on this theme, the authors report on findings from an impact evaluation of StreetSafe Boston, a streetwork intervention providing violence mediation and social services to selected members of 20 Boston gangs from 2009 to 2014. Unlike some other studies in the field, the researchers relied on gang-level (rather than neighborhood-level) measures in an effort to more directly link outcomes to the work of the program. The results were discouraging: The intervention was associated with an increase in shootings among the gangs served relative to comparison gangs.
This evaluation is not the final word on streetwork, of course, but it is an important cautionary note, casting doubt that this politically popular intervention is a silver-bullet solution to the problem of gang-related violence.
The Bottom Line: “Policymakers must understand that [streetwork] programs cannot be depended on to immediately reduce violence — and can produce harm — despite their persuasive policy framing. Observers that have held up the efficacy of streetwork models as an argument for immediately reallocating social resources dedicated to responding to violence are — for the moment, at least — overly optimistic. For existing streetwork practitioners, our message must be more urgent. Although outreach workers can and do positively influence the lives of the people they serve, everyday street outreach practices often risk increasing client exposure to violence. The existing evidence — built on a broad base of criminological study — persuades that leaders in the field of streetwork should commit to moving programs away from risky group-based activities and streetworker practices that facilitate connection within and across networks exposed to violence.”
When police pull back: Neighborhood-level effects of de-policing on violent and property crime
Justin Nix, Jessica Huff, Scott E. Wolfe, David C. Pyrooz and Scott M. Mourtgos
This essay in Criminology zeroes in on 2020, annus horribilis, a time defined by COVID-19, civic unrest and increased crime across the United States. The research team notes that in 2020, the U.S. experienced the largest annual increase in homicides ever recorded. (One of the scholars prominently cited in this piece is criminologist Rick Rosenfeld, who did so much to sharpen discussions of national crime trends. Sadly, Rosenfeld passed away in January 2024.) Many analysts have argued that the surge in crime in 2020 should be attributed to reduced policing, as both pandemic restrictions and racial justice protests caused police to be less active than usual. On the other hand, it is a common refrain in activist circles that police don’t stop or prevent crime, so there is also an argument that the rise in violence had nothing to do with policing.
Did the police pull back in 2020? And, if so, did the changes in police behavior have an impact on crime? In an effort to answer these questions, the research team examined what happened in Denver, Colorado, in 2020. They conclude that Denver police officers engaged in significantly fewer discretionary activities in 2020 in response to COVID-19 and the George Floyd protests. For example, police made 11,150 pedestrian stops in 2020 — down 50% from an average of more than 22,000 in the four preceding years. (Other measures of police activity, like vehicle stops and disorder arrests, also saw reductions.) Denver experienced an additional 528 violent crimes and 7,203 property crimes in 2020 — increases of 14% and 27%, respectively.
Recognizing that they have delivered results that may challenge the beliefs of some of their readers, the authors conclude on a plaintive note, offering the hope that future discussions of policing will be “guided by sound evidence rather than by emotion because the livelihoods of the public are at stake.”
The Bottom Line: “(De-)policing impacts crime. Reduced stops were associated with more violence in most Denver neighborhoods. Reduced drug arrests were likewise associated with increases in property crimes. Meanwhile, changes in disorder-related arrests were unrelated to changes in crime. These findings are consistent with the proactive policing literature and suggest that surgical use of police stops may provide beneficial impacts in the form of crime reductions.”
Why Americans Are a People of Exceptional Violence
Michael Tonry
Published in the journal Crime and Justice, this review essay by the eminent criminologist Michael Tonry combs through the data to make a point that is hiding in plain sight: The United States is an exceptionally violent place. Tonry highlights an important wrinkle, namely the difference between the U.S. and other Western nations is in the seriousness of the violence here, not the prevalence. According to Tonry, victimization rates for assaults in the United States are comparable to those in other English-speaking countries. But the presence of guns means that violence has a greater chance of becoming lethal in the United States in a number of circumstances, including robberies and domestic conflicts.
Moving from data to theory, Tonry offers a number of hypotheses to explain the high rates of violence in the U.S. Like many critics, he spotlights the ongoing effects of America’s pernicious history of racism, which has hardened many white Americans to the plight of Black Americans in particular. He also bemoans the easy availability of firearms. More provocatively, Tonry also calls attention to cultural factors, arguing that the history of westward expansion in the United States helped to shape a set of “frontier values” — such as rugged individualism, libertarianism, fatalism and resentment of outside authority — that contribute to contemporary violence.
The Bottom Line: “Only in the United States among Western countries are ordinary citizens routinely allowed to possess handguns and semiautomatic firearms, weapons designed not for hunting animals but for killing or injuring other people, and to carry them in public. In no other Western country are police killings or mass shootings so common. In no other Western country are people commonly sentenced to decades-long prison sentences or for life, punishments that make it nearly impossible ever again to live a normal life. No other Western country retains capital punishment.”
Labor Market Impacts of Reducing Felony Convictions Amanda Y. Agan, Andrew Garin, Dmitri K. Koustas, Alexandre Mas and Crystal Yang
In recent years, significant energy has been devoted to identifying and alleviating the “collateral consequences” of conviction and incarceration. Activists have been particularly focused on the way that time behind bars can affect a person’s employment trajectory. In 2014, California passed Proposition 47, allowing some felonies to be reclassified as misdemeanors. Would the proposition help increase employment by allowing people to honestly say on job applications that they did not have a felony conviction?
In an effort to answer the question, the research team focused on San Joaquin County, where the local public defender filed petitions on behalf of Proposition 47-eligible defendants without requiring their participation. Between December 2014 and September 2019, this effort yielded over 8,000 successful reduction petitions.
Using a variety of methods, including a randomized controlled trial, the team came to the conclusion that there was no positive impact on employment outcomes for the individuals who had benefited from the downgrading of felonies to misdemeanors. However, consistent with prior research, when the researchers looked at the outcomes of a subset of individuals who had self-initiated their petitions for a reduction, they did find evidence of a “marginally significant 12% increase in the probability of having any wage employment in the year of the reduction,” suggesting that future researchers should be mindful of the possibility of selection effects. The findings should also remind us of the need for modest expectations when it comes to criminal justice reform initiatives.
The Bottom Line: “We find little evidence that employment, self-employment and tax-filing outcomes improve after a public defender initiated reduction from a felony to a misdemeanor.”