Lifting the Veil on the Design of Predictive Tools in the Criminal Legal System

Recently, the National Institute of Justice (NIJ) — the research arm of the Department of Justice (DOJ) — put out a call for researchers to participate in what they called the “Recidivism Forecasting Challenge”. The challenge was designed to use information about people on parole in Georgia to “improve the ability to forecast recidivism using person-and place-based variables,” encourage “non-criminal justice forecasting researchers to compete against more ‘traditional’ criminal justice researchers,” and provide “critical information to community corrections departments.” Challenge contestants were awarded a collective total of $723,000 for their submitted models.

While heralded by the NIJ as a successful effort that “demonstrate[d] the value of open data and open competition,” in reality, the challenge was marked by serious and fundamental flaws. One of the winning papers encapsulated the issues with the challenge best when they said, “We are hesitant to accept any insights gained from submitted models and question the reliability of their performance. We would also discourage the use of any submitted models in live environments.” Six of the other 25 winning papers also expressed their concerns about the use of models created for the challenge in real-world environments.

So, what contributed to the challenge’s failures?

We argue in a new research study critiquing the challenge that a failure to engage impacted communities (those whose data was used for the challenge) as well as public defenders and other advocates for impacted communities contributed in part to some of the failures of this project. The standard going forward for developing predictive tools should draw on recent resources from the federal government to inform decision-making around whether to develop predictive tools. These efforts should center around developing strong protections for the people whose data is used to build automated systems and the people who may ultimately be evaluated by those systems if they are deployed.

Challenge Accepted? A Critique of the 2021 National Institute of Justice Recidivism Forecasting Challenge

Source: American Civil Liberties Union

So, why does this matter?

The NIJ has a lot of power, given its position within the Department of Justice, to shape the way that local community corrections departments think about recidivism. We submitted a Freedom of Information Act request to the DOJ to try to better understand how the results of the challenge have been or will be used but have not yet received a response to our request. While it is not fully clear yet how the results of the challenge will be used by the DOJ, the NIJ has already signaled that these types of tools are important to it by spending close to $1 million creating and executing the challenge. Furthermore, the DOJ, through the Bureau of Prisons, already uses a risk assessment tool, PATTERN, to make critical decisions about incarcerated populations. The use of this tool has been roundly criticized by several civil rights organizations.

Beyond influencing decisions about imprisonment and government surveillance, the data produced by law enforcement agencies and the predictions generated from risk assessment tools are often used in making decisions that can have a catastrophic impact on people’s lives — including loss of parental rights, homelessness, prolonged job insecurity, immigration consequences (including deportation), and inability to access credit. The voices of those impacted by these tools should be embedded in the design and implementation of these tools, as they are the individuals who will have to suffer the consequences of poorly designed systems. By involving impacted communities in the development of predictive tools, the design of these types of systems may look dramatically different, or these tools may be determined to not be useful at all.

For more information about the NIJ’s Recidivism Forecasting Challenge and its shortcomings, check out our paper below. Our paper was presented at the Association for Computing Machinery’s Conference on Equity and Access in Algorithms, Mechanisms, and Optimization at the end of October, where it won an Honorable Mention for the New Horizons Award.

We need you with us to keep fighting

Donate today

Trump's Attempt to Unilaterally Control State and Local Funding is Dangerous, Dumb, and Undemocratic

The Trump administration has not been subtle in its desire to use federal funding for political punishment. Whether threatening to cut off grants to sanctuary cities, to block financial assistance to states that push back against the president’s demands, or to freeze all federal grants and loans for social services across the country, Trump and his allies want us to believe they can wield the federal budget like a weapon. The reality is that the administration’s ability to withhold or condition funding is far more limited than they let on. The Constitution, Supreme Court precedent, and long-standing federal law stand firmly in the way of this brazen abuse of presidential power. Trump’s Attempted Funding Freeze? Blocked Immediately A week into his second administration, Trump attempted to freeze trillions of dollars in federal grants and loans that fund a vast array of critical services already approved by Congress. If allowed to go into effect, this unprecedented and far-reaching...

Documents Reveal Confusion and Lack of Training in Texas Execution

As Texas seeks to execute Carl Buntion today and Melissa Lucio next week, it is worth reflecting on the grave and irreversible failures that occurred when the state executed Quintin Jones on May 19, 2021. For the first time in its history — and in violation of a federal court’s directive and the Texas Administrative Code — Texas excluded the media from witnessing the state’s execution of Quintin Jones. In the months that followed, Texas executed two additional people without providing any assurance that the underlying dysfunction causing errors at Mr. Jones’ execution were addressed. This is particularly concerning given that Texas has executed far more people than any other state and has botched numerous executions. The First Amendment guarantees the public and the press have a right to observe executions. Media access to executions is a critical form of public oversight as the government exerts its power to end a human life. Consistent with Texas policy, two reporters travelled t...

The Supreme Court Declined a Protestors' Rights Case. Here's What You Need to Know.

The Supreme Court recently declined to hear a case, Mckesson v. Doe , that could have affirmed that the First Amendment protects protest organizers from being held liable for illegal actions committed by others present that organizers did not direct or intend. The high court’s decision to not hear the case at this time left in place an opinion by the Fifth Circuit, which covers Louisiana, Mississippi, and Texas, that said a protest organizer could be liable for the independent, violent actions of others based on nothing more than a showing of negligence. Across the country, many people have expressed concern about how the Supreme Court’s decision not to review, or hear, the case at this stage could impact the right to protest. The ACLU, which asked the court to take up the case, breaks down what the court’s denial of review means. What Happened in Mckesson v. Doe? The case, Mckesson v. Doe , was brought by a police officer against DeRay Mckesson , a prominent civil rights activi...

Search This Blog