JT/DL: Not Everything is Attributable to AI
Plus new jobs
The JT/DL is a twice-monthly newsletter about justice technology news, events, and opportunities. My opinions do not reflect those of my employers or professional partners.
Access to Justice in the Age of AI, a Response
Last we spoke, I argued that there’s little evidence that justice tech helps people. Now there’s a paper hinting that AI might be helping people get to court without attorneys (and win?)—but that isn’t what the authors conclude. Being that the piece received significant attention, it deserves a closer look.
I’m referring to a working draft of an academic paper, “Access to Justice in the Age of AI: Evidence from U.S. Federal Courts”, by Shah and Levy. A New York Times article from late May amplified its findings. The paper’s core claim is that the share of federal filings by people without attorneys—so-called self-represented litigants (SRL)—grew from 11% to 17% between 2020 and 2025. AI, the authors conclude, is driving the increase.
The problem is that the paper can’t prove its conclusion. Are people using AI to write court filings? Of course: everyone is using AI to write everything. The problem is that the paper went further than its data can support. While grabbing AI as the explanation, it ignored historical drivers of SRL litigation, like the economy. It relies on AI-detection software, which is a suspect research method. And what’s worse: in framing SRLs as a burden on courts, it blames people for using the one resource they can afford to help navigate the justice system, while letting corporate culprits off the hook.
Not Everything is Attributable to AI
The data at the center of the paper are 1,600 SRL court filings from before and after the public release of ChatGPT in 2023. They write that “AI-generated text has been rising monotonically [in SRL federal filings] from 1.0% in 2023 to 18.0% in early 2026.” At the same time, SRL filings as a total of federal filings are up 6%. The authors float two “descriptive” conclusions: AI is creating new litigants who otherwise wouldn’t have sued, or it’s convincing people to drop their lawyers and go at it alone. They treat both as equally plausible without interrogating either.
The upward trend in SRL and represented federal filings started in 2021 after the pandemic dip and two years before ChatGPT’s public launch. While the growth curve gets steeper in 2024 and 2025, the pre-existing trend matters because the period from 2021 onward was one of escalating economic distress for American households. By the end of 2025, total household debt grew to $18.8 trillion. Credit card balances rose to $1.28 trillion. Delinquency rates on auto loans and credit cards climbed steadily, particularly among lower-income borrowers. People who are falling behind on debts get sued, and people who can’t pay a $2,000 credit card bill can’t afford a lawyer.
The paper’s own framing reinforces this point, though the authors don’t draw the connection. Section 3.3 sorts cases into “simple” (consumer debt, civil rights, foreclosure) and “complex” (securities fraud, patent infringement). They note that the SRL uptick is concentrated in the simple case categories. The authors treat this as evidence that AI helps with cases that require less lawyer intervention. But they never consider that simple and complex cases, as they define them, impact different demographics. The average salary of a patent holder in the U.S. is $256,000. Securities ownership is dominated by households earning more than $100,000 a year. Both groups have access to resources, which buys legal representation. By contrast, we know economic hardship is a driver of litigation. Even the authors’ own data shows a 42% increase in SRL filings in federal court between 2008 and 2012. During that time, SRLs went from 10% of the federal docket to 14% (remember, the authors’ big statistic is that AI caused a 6% jump in SRL filings). This is all to say, the growth in SRL cases is at least as consistent with growing economic hardship as it is with AI adoption.
Beyond the economy, national politics and federal priorities also deserve consideration. The single largest year-on-year jump in SRL filings is in 2025. The Times article quotes a federal court staff attorney in Minnesota saying he’d seen a “50 percent uptick in filings from non-prisoners starting around March 2025.” By then, we were two months into an administration abusing the public and dismantling the federal agencies that traditionally filed suits in court on behalf of people who can’t afford to. By October 2025, the Trump-led Consumer Financial Protection Bureau had dismissed or rolled back 42 enforcement actions and gutted its consumer complaint program. The DOJ’s Civil Rights Division froze all new litigation and lost an estimated 70 percent of its attorney staff—a fraction of the over 10,000 attorneys that have left federal service in the last 18 months.
When federal enforcement retreats, bad actors face fewer consequences, harmful practices increase, and individuals are left to fend for themselves. The paper simply doesn’t consider the economic and political forces that have driven people into court without counsel for decades.
AI Detection Isn’t
Setting aside alternative explanations, the paper’s methodology deserves scrutiny.
The study relies on Pangram, a proprietary black-box algorithm, to determine whether a filing contains AI-generated text. The authors cite one working paper that benchmarks Pangram’s accuracy. It’s a thin foundation, and the problems compound from there.
There is little published research on Pangram specifically, but AI detection tools have well-documented defects. Researchers out of Dublin concluded that benchmark accuracy is not a reliable measure of AI detection tools' real-world efficacy. These tools are benchmarked on domain-specific data, and accuracy drops when the tools are pointed at different subject matter. The benchmark cited by the authors wasn’t tested on legal text, a significant gap for a paper about court filings.
Another red flag is that false positives are a chronic problem. Stanford researchers found that non-native English speakers’ writing is disproportionately flagged as AI-generated. A finding with obvious implications for the demographics that make up SRL litigants. False negatives are equally concerning. When AI text is paraphrased, detection accuracy dropped from 70.3% to 4.6% in one study. Other research found that hybrid texts—documents mixing human and machine-generated writing—also cause accuracy to drop.
Taken as a whole, these tools offer a limited signal, not a verdict. These weaknesses in the methodology are particularly important to call out because the authors’ first recommendation is to replicate this study on state court data. State court dockets are where the access-to-justice crisis is most acute, and replicating this methodology would misidentify who is filing and why. Building policy on top of such a weak foundation would do real harm to the most disadvantaged people trying to navigate our justice system.
Stop Blaming the Poor
Even if the methodology were stronger, the paper’s framing raises a separate concern: it blames people without attorneys as the source of court strain.
Starting in the abstract and running throughout, the authors frame increased SRL filings as a “burden” on the courts. However, they do not show that an increase in AI use has led to weaker cases. Well before AI, SRLs had a bad track record winning in court. However, the authors’ own data says SRLs are losing 6% less than before ChatGPT went public and getting settlements 3% more of the time. Whether these changes are attributable to AI is unclear, however, and they deserve more attention. But, if SRL outcomes are improving, the “burden” argument gets harder to make.
Second, the paper glosses over its own data showing that filings by represented parties are up nearly 8% over the 2020-2025 period. Other studies provide evidence from state courts that AI is being used by corporate filers, particularly in America’s fastest-growing docket: debt collection. A handful of companies drive the majority of predatory debt litigation that looks to win by default, not on the merits of the claim. If AI-powered bulk filing is left unchecked, it will create systemic burdens and supercharge courts into a platform for wealth redistribution going the wrong direction. But that’s due to corporate greed, not poor folks.
American justice is already out of reach for lower and increasingly middle-class people. Publishing and elevating research that obscures this systemic problem by blaming the disadvantaged isn’t just wrong, it’s harmful. The crisis isn’t that people have found a tool that helps them get to court, it’s that it took this long.
News
AI found in court filings. (New York Times)
Florida’s Supreme Court amends rules to address AI use in court filings. (Florida Bar)
California judges are testing a new AI clerk, and you won’t know if it’s looking at your case. (CalMatters) (h/t Keith Porcaro)
A company put AI cameras in thousands of school buses, now they want to give cops access. (404 Media)
Crime mapping in the AI era: why transparency still matters. (Biometric Update)
China is testing its state surveillance model abroad. (New York Times)
Nashville Police conduct “Drone as First Responder” trial program. (WSMV) (h/t Brittany Suszan)
The expungement experiment: impact on employment. (A2J Lab) (h/t Tanina Rostain)
£50m research and innovation program launched to make Britain’s streets safer for everyone. (UKRI) (h/t Erin Szulman)
Events
The NYC Leadership Summit on AI in Criminal Justice is June 16. (JJC)
Wikimania will be in Paris July 23-25. (WM)
The A2J Network Conference will be in Cincinnati October 21-22. (A2JN)
Jobs & Opportunities
[New] Anthropic needs someone to tackle AI and the rule of law. (A\)
Arnold Ventures is looking to fill multiple roles. (AV)
The Brennan Center for Justice has multiple openings. (BCJ)
The Center for Democracy and Technology has academic externships. (CDT)
The Chan Zuckerberg Initiative needs a counsel for AI and tech. (CZI)
The University of Chicago Crime Lab has multiple openings, including internships. (UCCL)
Code for America has multiple openings. (CfA) (h/t Russ Finkelstein)
The Kapor Foundation needs research fellows. (KF)
The Legal Aid Society Digital Forensics Unit needs a digital forensics analyst and a staff attorney. (LAS) (h/t Jerome Greco)
Maryland Legal Services Corporation needs a director of strategic technology. (MLSC) (h/t Dave Pantzer)
OpenMinded has multiple openings. (OM)
The Pew Charitable Trusts needs a data and policy officer for their courts work. (Pew)
Recidiviz has multiple openings. (R)
The Recoding America Fund needs a federal policy director. (RAF)
[New] Renaissance Philanthropy is hiring for multiple roles. (RP)
Stanford Law School needs an associate director for its AI initiative. (SLS)
[New] TechCongress opened it’s applications for its fellowship program. (TC)