The hallucination tax: what unverified AI citations really cost

There is a tidy excuse making the rounds whenever a brief turns up citing cases that do not exist: the AI did it. The model “hallucinated,” the black box misled me, the technology is new and imperfect. It is a comforting story, and it is wrong. Large language models are probabilistic text engines, not search tools — they predict the most plausible-looking sequence of words, which means a fabricated citation is not a malfunction but the expected behavior of a system asked to do something it was never built to do.^[1] A lawyer who treats a general-purpose chatbot as a legal database has not been ambushed by a glitch. They have skipped the one step the profession has always required: reading the authority before signing the filing.

That failure has a price, and someone always pays it. We call it the hallucination tax — the real, compounding cost that unchecked AI output imposes on litigation and, increasingly, on arbitration. It is paid in monetary sanctions, in the billable hours of opposing counsel and the tribunal, in delay, and most durably in lost credibility. None of it is recoverable from the model.

The duty did not change; the tooling did

Nothing in the rules of professional conduct treats AI as a special case, because nothing needs to. ABA Model Rule 1.1 requires competent representation, and Comment 8 — added in 2012 — directs lawyers to keep abreast of “the benefits and risks associated with relevant technology.”^[2] Rule 5.3, retitled the same year to cover nonlawyer assistance rather than merely assistants, extends a supervising lawyer's responsibility to non-human help — which now plainly includes generative tools.^[2] In July 2024 the ABA made the point explicit in Formal Opinion 512: a lawyer's “uncritical reliance on” AI output without “an appropriate degree of independent verification or review” can breach the duties of competence and candor.^[3]

Federal Rule of Civil Procedure 11 says the same thing in older language. By signing a paper, an attorney certifies that the legal contentions are warranted by existing law, after an inquiry “reasonable under the circumstances.”^[4] Citing a case you never read — because a model invented it — is the textbook failure of reasonable inquiry. The gatekeeping obligation is non-delegable. You cannot hand it to a summer associate, and you certainly cannot hand it to a chatbot.

The case law of consequence

The jurisprudence matured fast, from cautionary tale to hard penalty. The origin point is Mata v. Avianca, Inc., where plaintiff's counsel filed a brief citing multiple nonexistent opinions generated by ChatGPT, then — when challenged — submitted fabricated “copies” of the phantom decisions.^[5] Judge Castel imposed a $5,000 sanction, but the holding's value is in its framing: “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance,” the court wrote, while stressing that “existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.”^[5] The wrong was not the tool. It was abandoning the gate.

If Mata was a warning, Park v. Kim showed the appellate teeth. There the Second Circuit confronted an attorney whose reply brief cited a case she admitted was generated by ChatGPT and that did not exist; the court referred her to its grievance panel.^[6] And the costs are no longer symbolic. In Lacey v. State Farm General Insurance Co. (C.D. Cal. 2025), a special master who candidly admitted he “almost” adopted the fake citations into an order imposed roughly $31,000 in sanctions — about $26,000 for the work of reviewing the briefs and holding the hearing, plus $5,000 to reimburse opposing counsel.^[7] Even where courts decline to punish, the spectacle lingers: when former attorney Michael Cohen's submission carried fake cases produced by Google Bard, Judge Furman declined sanctions but memorialized that, “[a]s far as the Court can tell, none of these cases exist.”^[8] Surviving sanctions is not the same as surviving the record.

Why arbitration pays the tax twice

The economics bite harder in arbitration than in court. A tribunal is paid by the hour, so every hour spent chasing a citation that does not exist is billed straight to the parties. Worse, arbitration's defining promise — speed, economy, finality — runs on trust. The first fake case an arbitrator catches converts a sympathetic reader into a forensic skeptic who now distrusts every factual assertion in the brief. Under a loser-pays regime, submitting hallucinations is close to writing your opponent a blank check for the cost of catching them.

Then there is enforcement. If a tribunal relies, even inadvertently, on a fabricated legal principle, the resulting award becomes vulnerable: a losing party can argue it was unable to present its case against fictitious law, or that the award offends public policy — grounds that echo the New York Convention's narrow but real bases for refusing enforcement. A hallucinated brief is a poison pill in your own client's victory, handing the loser a fresh argument for years of post-award litigation.

The institutions have noticed. JAMS issued dedicated Artificial Intelligence Disputes Rules effective April 2024, the first comprehensive ADR framework for AI matters.^[9] The AAA-ICDR followed with March 2025 guidance on arbitrators' use of AI tools, updated later that year, instructing that AI outputs be critically evaluated and cross-referenced against primary sources, with independent decision-making retained and material AI use disclosed.^[10] The throughline is unmistakable: AI is a permitted assistant, never a substitute for verification.

Verify before you file

The defense against the tax is unglamorous and entirely within the filer's control. Three habits do most of the work:

1. Ground the research, then click through. Use general-purpose chatbots for brainstorming and drafting prose — never for generating citations. Reserve citation work for tools tied to primary law, and let no authority into a filing until someone has opened the actual source and confirmed the case exists, the pin cite is right, and the opinion says what the brief claims.
2. Keep a verification log. A short record showing that every authority was independently checked is cheap insurance — it both prevents the error and rebuts any later charge of bad faith. See /research-lab.
3. Run the table of authorities before filing. Push every cite through a traditional database; anything that fails to resolve or flags as unrecognized halts the filing until it is manually retrieved or removed.

The black box is, in the end, a mirror. When it produces garbage and that garbage gets filed, what shows up in the record is not the model's unreliability but the filer's. The mandate for the AI era is the oldest one in the practice, only more so: trust, but verify. Anything less is not innovation — it is the hallucination tax, and the client pays it.