Numbers game

Two people with identical criminal records — or lack thereof — are convicted of the same crime. A judge sentences one to probation, while the other is incarcerated because that person is determined to have a higher risk of recidivism.

Judicial discretion is a tenet of the American justice system, giving the judge in this hypothetical situation the right to impose whatever sentence she deems appropriate. But what if the judge’s decision-making process was informed by an algorithm? And what if the algorithm wasn’t intended to be used for sentencing in the first place? What if the situation wasn’t hypothetical? It’s not.

Richmond Law professor Erin Collins understands the inherent logic in actuarial sentencing, as the practice is called. In theory, the data provided through risk assessment instruments can help judges curb their own biases, benefiting low-risk offenders.

“As a country, I think we’re coming to a reckoning with mass incarceration across the political spectrum, agreeing that we need to change the way we’ve been administering our criminal justice system, the way we’ve been punishing people,” Collins said. “Increasingly we’re looking to data-driven, evidence-based approaches to help us solve these tricky dilemmas.

“I think it’s attractive for a lot of reasons,” she added. “It seems to be objective. It seems to be kind of unassailable if it’s based on an empirical analysis. How could that be wrong?”

For Collins, that question isn’t rhetorical. As she notes in “Punishing Risk,” her 2018 article in Georgetown Law Journal, statistical predictions of the likelihood of future criminal behavior are far more complicated than simply running the numbers.

Take the 2016 case of State v. Loomis, in which Eric Loomis of La Crosse, Wisconsin, was arrested for driving a stolen car and accused of being the driver in a drive-by shooting. While he denied being involved in the shooting, Loomis pleaded guilty to operating a motor vehicle without consent and eluding a police officer. He was sentenced to six years in prison.

The judge in his case divulged that Loomis’ algorithm-driven risk assessment, which indicated he had a high risk of recidivism, factored into the sentencing decision. Loomis’ appeal to the Wisconsin Supreme Court was denied.
Some risk-assessment instruments consider a wide variety of factors, such as socioeconomic status, that wouldn’t otherwise be seen as a legitimate part of the sentencing equation. Those factors can be used against people who share traits with groups of past reoffenders.

“What these tools are really saying is, ‘Other people who have characteristics like you have or have not offended at a high, moderate, or low rate,’” Collins said. “Then, they juxtapose what people generally in the aggregate have done who have these characteristics onto this one person to make a prediction about this person’s likelihood of recidivism.”

Increasingly used in jurisdictions around the country, actuarial sentencing informed by risk assessment instruments — typically surveys — doesn’t consider the fact that the tools themselves can have some of the same biases that judges would likely face criticism for harboring. There’s also the fact that the instruments were initially designed to be used for programming and classification purposes in correctional facilities, meaning that using them as recidivism predictors for sentencing purposes is a non-prescribed (or as Collins refers to it, “off-label”) usage.

“If the argument is, ‘Judges should be doing this anyway,’ I think we should be saying, ‘Really? Are you sure about that?’” Collins said. “My concern — this is not an empirical claim — is that this is going to lead us to keep replicating the same disadvantages and disparities that we already have in our current system.

“The way risk assessment is talked about is it’s only going to lead to good consequences for people,” she continued. “My concern is that it could work that way, but it could also work the other way, which is to keep people who are deemed high risk in prison for longer or to send them to prison or jail instead of letting them serve their sentences in the community. It’s not necessarily just a one-way tool that will just lead to one result.”

The issues surrounding actuarial sentencing are complex, but Collins and other legal scholars are addressing them now, before they become a commonplace, widely accepted part of the justice system. The following essay provides some insight into why there is apprehension about its increasing adoption.

The Perils of ‘Off-Label Sentencing’
By Erin R. Collins

Current criminal justice reform efforts are risk-obsessed. Actuarial risk assessment tools, which claim to predict the risk that an individual will commit, or be arrested for, criminal activity, dominate discussions about how to reform policing, bail, and corrections decisions.

And recently, risk-based reforms have entered a new arena: sentencing.

Through the practice of “actuarial sentencing” (also called “evidence-based sentencing”), jurisdictions across the country are allowing or requiring sentencing judges to consider the recidivism predictions of actuarial risk assessment tools.

An actuarial risk assessment tool is essentially a structured survey that inquires whether an individual has certain “recidivism risk factors,” or characteristics that statistically correlate with recidivism. The individual is scored points for the presence or absence of these factors, and based on the individual’s total score, he or she is deemed to pose a low, medium, or high risk of recidivism.

Actuarial sentencing has gained the support of many practitioners, academics, and prominent organizations, including the National Center for State Courts and the American Law Institute.

This enthusiasm is, at first blush, understandable: actuarial sentencing seems to have only promise and no peril. It allows judges to identify those who pose a low risk of recidivism and divert them from prison. Society thus avoids the financial cost of unnecessarily incarcerating low-risk individuals.

And yet, this enthusiasm for actuarial sentencing ignores a seemingly crucial point: Actuarial risk assessment tools were not developed for sentencing purposes.

In fact, the social scientists who developed the most popular risk assessment tools specified that they were not designed to determine the severity of a sentence, including whether or not to incarcerate someone. Actuarial sentencing is, in short, an “off-label” application of actuarial risk assessment information.

As we know from the medical context, the fact that a use is “off-label” does not necessarily mean it is ill-advised or ineffective. And, indeed, many contend that actuarial sentencing is a simple matter of using data gleaned in one area of criminal justice and applying it to another. If we know how to predict recidivism, why not use that information broadly?

Isn’t this a prime example of an approach that is smart — rather than tough — on crime?

As I contend in my article, “Punishing Risk,” which was published in Georgetown Law Journal, the practice of actuarial sentencing is not that simple, nor is it wise. In fact, using actuarial information in this “off-label” way can cause an equally unintended consequence: It can justify more, not less, incarceration — and for reasons that undermine the fairness and integrity of our criminal justice system.

The actuarial risk assessment tools that are being integrated into sentencing decisions, such as the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) tool and the Level of Services Inventory-Revised (LSI-R), were designed to assist corrections officers with a specific task: how to administer punishment in a way that advances rehabilitation.

They are intended to be used after a judge has announced the sentence. They are based on the Risk-Need-Responsivity principle, according to which recidivism risk is identified so that it can be reduced through programming, treatment, and security classifications that are responsive to the individual’s “criminogenic needs” (recidivism risk factors that can be changed).

Sentencing judges, in contrast, do not administer punishment but rather determine how much punishment is due. In doing so, they may use actuarial risk predictions to advance whatever punishment purpose they deem appropriate. While they may decide to divert a low-risk individual from prison in order to increase their rehabilitative possibilities, they may also decide to sentence a high-risk individual more harshly — not because doing so will increase the prospects for rehabilitation, but because it will increase public safety.

Judges may use risk prediction to incapacitate, not rehabilitate. 

This conclusion begs the question: Don’t we want judges to consider an individual’s risk level when determining a sentence? Quite frankly, no — at least not in the way these tools measure and define risk.

The tools measure risk based on a range of characteristics that are anathema to a principled sentencing inquiry, such as gender, education and employment history, and family criminality. Perhaps consideration of these factors makes sense if the predictive output is used to administer punishment in a way that is culturally competent and individualized.

But in the sentencing context, it allows the judge to punish someone more harshly based on a compilation of characteristics that are inherently personal and wholly non-culpable and often replicate racial biases that pervade other areas of the criminal justice system. In other words, actuarial sentencing allows judges to defy the well-established tenet that we punish someone for what they did, not who they are.

Moreover, risk assessment tools define recidivism risk broadly. Many define it as the likelihood an individual will be convicted of a crime of unspecified severity in the proceeding years. Others are even less precise, predicting the likelihood that a person will be arrested for any reason (regardless of whether the arrest is justified or results in charges).

Thus, the question these tools answer is: How likely is it that a person with these characteristics will commit or be arrested for any type of criminal activity — including low-level nonviolent activity — in the coming years? Even if we accept that judges should be mindful of public safety in determining a sentence, these tools do not advance that inquiry.

Incorporating these tools into sentencing conflates recidivism risk, broadly defined, with risk to public safety. If we want to reduce our reliance on public safety, we must refine — rather than expand — the risk that counts for sentencing purposes.

Thus, it is not clear that these correctional tools advance a sound sentencing inquiry. But even if they did, it is questionable whether they actually enhance the accuracy of the predictions judges would make in their absence.
In fact, computer scientists recently found that people with little or no criminal justice expertise were able to predict recidivism at the same level of accuracy — approximately 65 percent — as a popular risk assessment tool.

One thing is clear: Many recidivism risk factors are markers of relative structural disadvantage and reflect historically biased criminal justice practices.

Thus, even if actuarial sentencing benefits people who are sufficiently low-risk to be diverted from prison, that benefit will not be evenly or fairly distributed amongst the defendant population. And those who are deemed a high risk based on these same factors may be more likely to be incarcerated, and perhaps for longer periods.

As I conclude in “Punishing Risk,” as top criminal justice policy makers call for a revival of the war on crime, “[N]ow, more than ever, we [must] carefully scrutinize how data is incorporated into criminal justice decisions, with particular attention to how we label people as ‘risky,’ and the consequences of that label.”