In February 2025 an associate at Morgan & Morgan uploaded a draft motion to the firm’s in-house AI platform and typed one sentence: “add to this Motion in Limine Federal Case law from Wyoming setting forth requirements for motions in limine”. Eight of the nine cases that came back did not exist. A partner and local counsel signed without reading, and Judge Kelly Rankin sanctioned all three.
Damien Charlotin’s database of court decisions involving hallucinated material stood at 2,039 entries on 12 September 2026, 811 of them involving lawyers. The legal prompting mistakes behind those orders are not exotic. They are the same dozen, repeated, mostly by competent lawyers who did not know that the words they typed were an invitation to invent.
Here are the twelve, each with the actual prompt from the record and a fix you can paste. The prompting hub covers the positive craft; this is the list of things not to type.
Mistake 1: “Add case law”, the prompt that produced eight fake cases
The prompt fails structurally. It asks the model to supply authority from memory, and a model’s memory is a compressed recollection of its training data in which a case name that “sounds right” is exactly what comes out; why AI makes up fake cases explains the mechanism. The fix reverses the flow: you supply the authority, the model works with it.
Draft only section [II.B] of a [motion in limine] arguing that [proposition] under [the applicable rule].
Use exclusively these authorities, which I have verified in [Westlaw/Lexis] today:
<authorities>
[case name, citation, pinpoint, one-line holding, one per line]
</authorities>
Structure: thesis; rule with citation; application to <facts>[facts]</facts>; the strongest counter-argument answered in two sentences; conclusion. Maximum [400] words.
Cite nothing outside the list. Where the listed authorities do not support a step, write [GAP: needs authority for X] rather than filling it.Mistake 2: leaving “[cite]” for the model to fill
Jeff Hancock is a Stanford professor and a credentialed expert on misinformation. Defending Minnesota’s political-deepfake law, he drafted an expert declaration with GPT-4o and typed “[cite]” where references should go. The model, in the court’s words, “apparently took ‘cite’ as an instruction and fabricated citations”: two non-existent articles and a third with the wrong authors. Judge Laura Provinzino excluded the declaration: Hancock had “fallen victim to the siren call of relying too heavily on AI — in a case that revolves around the dangers of AI, no less” (Kohls v. Ellison, D. Minn., 10 January 2025).
A bracket that means “I will fill this later” to you means “fill this now” to a model, which reads every token in the window as material to act on.
Mistake 3: asking for a “properly formatted citation”
This one caught Latham & Watkins, acting for Anthropic. An associate asked Claude “to provide a properly formatted legal citation” for a real article in an expert declaration. Link, volume, pages and year came back right; author and title were invented. The firm’s “manual citation check did not catch that error”, and Judge Susan van Keulen saw “a world of a difference between a missed citation and hallucination generated by AI” (Fortune, May 2025).
“Format this” sounds clerical; to the model it is generation, and it generates the fields it is missing. Paste the complete string and permit only rearrangement.
Reformat the citation below into [Bluebook 21st ed. / OSCOLA] form. Change only punctuation, ordering, abbreviations and typeface. Do not add, remove or alter any author, title, party name, year, volume, page or court. If a required element is missing, write [MISSING: element] instead of supplying it.
Citation as I have it:
[paste the full citation exactly as it appears in the source]Mistake 4: no jurisdiction
Clio’s prompting guide: “Jurisdiction is the piece most lawyers skip because it’s obvious to them. You’ve been thinking about Texas employment law all morning. The AI hasn’t.” Its gut check: if the same prompt could apply to a range of different matters without changing a word, it is too broad (Clio). “Is this non-compete enforceable?” gets you whichever law dominates the training data.
Assess the enforceability of the non-compete in Section [7.2] of the attached agreement under [Texas] law as of today. The employee is a [sales director] based in [city], employed [4 years], with access to [customer pricing and pipeline data]. The clause runs [24 months] and covers [the United States].
Apply only [Texas] law. Identify the controlling test and the factors a [Texas] court would weigh, and note any 2024-2026 developments, including the status of the FTC non-compete rule. Tag every authority [VERIFY]; I will check each in a primary database.Mistake 5: four tasks in one prompt
Clio’s own bad example: “summarise this deposition, identify credibility issues, compare it to the plaintiff’s affidavit, and draft follow-up questions”. The model “does none of them particularly well”. Justia explains the related failure: “If you ask it to draft, critique, and rewrite simultaneously, the model’s performance degrades. It will often rush the initial draft just to get to the critique” (Justia Onward). One task per turn: extract, then judge, then draft, with the critic told not to rewrite yet.
Mistake 6: trusting the first output
In Lacey v. State Farm (C.D. Cal., 2025) an outline produced with CoCounsel, Westlaw Precision and Gemini was folded, unread, into a brief filed by Ellis George and K&L Gates; nine of 27 citations were wrong and the two firms paid $31,100. The special master: “I read their brief, was persuaded (or at least intrigued) by the authorities that they cited, and looked up the decisions to learn more about them — only to find that they didn’t exist. That’s scary.”
In Noland v. Land of the Free (Cal. Ct. App. 2025) Amir Mostafavi ran ChatGPT-“enhanced” briefs through other AI tools instead of reading them; 21 of 23 quotations were fabricated, the sanction $10,000. GC AI’s tracker reduces every entry in the sanctions timeline to one sentence: “Every lawyer in this tracker trusted an output they had not read.” The fix is not a prompt. As the Singapore Academy of Law’s guide puts it, do not “Expect perfect output on a single try”: run it twice, compare, then read.
Mistake 7: client facts in a consumer tool
In Morgan v. V2X (D. Colo., 30 March 2026) the court amended a protective order so that, in Clio’s summary, confidential information may not go into “any mainstream AI tool like standard ChatGPT, Claude, Gemini, or similar platforms” unless the provider’s terms protect it. In United States v. Heppner (S.D.N.Y., February 2026) Judge Rakoff held a defendant’s consumer-Claude conversations unprotected: “Because Claude is not an attorney, that alone disposes of Heppner’s claim of privilege.” The UK Upper Tribunal says putting client letters into ChatGPT “is to place this information on the internet in the public domain”.
The habit that avoids all three is Brooke Loesby’s, from the ABA Journal: not “My client Sarah is suing her business partner for embezzling $400,000” but “I am working on a partnership dispute involving allegations of financial misconduct.” The abstract version still yields causes of action, elements and a document list. If the facts must go in, you need a no-training tier; is ChatGPT confidential for lawyers tells you which tier you are on.
Mistake 8: asking the chatbot whether the cases are real
Steven Schwartz did this. Ordered to produce the cases in Mata v. Avianca, he asked ChatGPT whether Varghese was real and was told it could “be found in reputable legal databases such as LexisNexis and Westlaw”. The firm had neither, only a limited Fastcase plan. Judge Castel read a fabricated opinion aloud: “Can we agree that’s legal gibberish?” The sanction was $5,000, jointly and severally (Mata v. Avianca, S.D.N.Y., 22 June 2023).
Regulators and courts now say it in terms: the Swiss bar’s guidance that you cannot simply ask an AI system whether its output is true (my translation), and New South Wales Practice Note SC Gen 23, para 17: “Such verification must not be solely carried out by using a Gen AI tool or program.” The model lists; you check, in a database, with a citator; how to verify AI legal citations is the routine.
List every case, statute, rule and secondary source cited in <document>[paste]</document> in a table: Citation as written | Type | Proposition it is cited for (quote the sentence) | Pinpoint given? (Y/N) | Quotation present? (Y/N).
Do not tell me whether any citation exists, is good law or supports the proposition; I will check each in [Westlaw/Lexis/BAILII] myself. Then list the quotations you would test first against the source, with reasons.Mistakes 9 to 12: one chat for everything, firm-name personas, vague verbs, and the model’s own limits
9. One chat for everything. Long conversations become “haunted by the ‘ghosts’ of past prompts”, in Justia’s phrase: a jurisdiction named for the last matter, a persona set an hour ago. Its rule: “Treat your chat sessions like individual case files.” The SAL/Microsoft guide’s first “do” is the same: “Start a new chat for each task.” Standing context, such as a playbook or house style, belongs in a Project or custom GPT with written instructions.
10. “You are a senior corporate attorney at Skadden Arps…” On 19 March 2026 a non-lawyer’s X thread offering “12 prompts that replace $15,000 in legal bills” went viral on exactly this persona formula. Artificial Lawyer asked the right question: “It’s possible that Claude simply read ‘Wachtell’ and thought ‘OK, this means do the contract in the style of any large commercial law firm’” (Artificial Lawyer, 20 March 2026). Anthropic’s own guidance says “heavy-handed role prompting is often unnecessary”. The persona changes tone; your side, the governing law, the reader and the bargaining position change the substance.
11. “Review this contract.” For nobody, against nothing, in no format. Harvey’s line: “Vague prompts produce generic answers. Structured prompts produce reviewable work product.” State side, playbook, jurisdiction and columns; drop the adjectives, which Thomson Reuters’ CoCounsel team warns “can be confusing or misleading” to a tool that “takes things literally”.
12. Ignoring what the model is bad at. Thomson Reuters lists “math, counting, and sorting” as known weak spots, so date arithmetic and totals go into Excel. A 200-page dump with the question at the top degrades recall. And habits are model-specific: OpenAI says that for its o-series reasoning models “prompting them to ‘think step by step’ or ‘explain your reasoning’ is unnecessary”; Anthropic says to remove “double-check your answer” from Claude Opus 5 prompts because it causes over-verification. See prompting reasoning models.
The fix sheet
| # | The mistake | The fix | Where it went wrong |
|---|---|---|---|
| 1 | “add case law from Wyoming” | Supply verified authorities; the model uses only those | Wadsworth (D. Wyo. 2025) |
| 2 | “[cite]” left in a draft | Brackets are yours; search for “[” before sending | Kohls (D. Minn. 2025) |
| 3 | “Format this citation” | Paste the full citation; permit rearrangement only | Concord (N.D. Cal. 2025) |
| 4 | “Is this non-compete enforceable?” | Name the law, the date and the facts; “apply only X law” | Clio prompting guide |
| 5 | Summarise, judge, compare and draft at once | One task per turn; separate drafter from critic | Clio; Justia |
| 6 | Outline folded into a brief unread | Run twice, compare, read all of it | Lacey; Noland |
| 7 | “My client Sarah is suing…” | Abstract framing, or a no-training tier | Morgan v. V2X; Heppner; UKUT 81 |
| 8 | “Are these cases real?” | The model lists; you check in a database with a citator | Mata (S.D.N.Y. 2023) |
| 9 | One chat for everything | New chat per task; Projects for standing context | Justia; SAL/Microsoft |
| 10 | “senior corporate attorney at Skadden Arps” | State side, audience, standard and bargaining position | Nav Toor thread; Anthropic |
| 11 | “Review this contract” | Side, playbook, jurisdiction, output columns | Harvey; CoCounsel team |
| 12 | Arithmetic in chat; step rituals on reasoning models; 200-page dumps | Excel for numbers; goal and success criteria; documents first | Thomson Reuters; OpenAI; Anthropic |
Where to go next: prompt engineering for lawyers is the positive version of this page; corrected prompts sit in ChatGPT prompts for lawyers and the prompt library; and if you have already found a bad citation in a filing, what to do when you find a fake citation is the same-day protocol. Every one of these mistakes is a live exercise in AI Lab for Lawyers: you make them in a sandbox so that you never make them in a filing.
Frequently asked questions
What is the most common ChatGPT mistake lawyers make?
Asking the model to supply legal authority it does not have in front of it. Prompts such as 'add case law' or 'what is the leading case on X' invite invention, and the invented case name will sound right. The second most common is not reading the output: every sanctioned lawyer in the public trackers signed something they had not checked. Supply the authorities yourself and read everything before it leaves your desk.
Can I ask ChatGPT to add case law to my brief?
No. That is the prompt behind Wadsworth v. Walmart, where eight of nine cases added by a firm's own AI platform were fabricated and three lawyers were sanctioned. Any general model answers 'add case law' from memory, and memory is where fake cases come from. Find the authorities in Westlaw, Lexis or an official database, verify them, paste them into the prompt, and instruct the model to use nothing else.
Why should I not ask the AI to format citations?
Because formatting is still generation. In Concord v. Anthropic a Latham & Watkins associate asked Claude for 'a properly formatted legal citation'; the link and page numbers came back right but the author and title were invented, and the firm's manual check missed it. If you want reformatting, paste the complete citation string and forbid the model from adding or changing any element; anything missing should be flagged, not supplied.
Does adding a law-firm name to the prompt improve the output?
There is no evidence that it improves accuracy. The viral March 2026 'senior corporate attorney at Skadden Arps' prompts changed the register of the drafting, and Artificial Lawyer's suspicion was that the model simply read the firm name as 'draft like a large commercial firm'. Anthropic's guidance says heavy-handed role prompting is often unnecessary. State the side you act for, the governing law, the reader and the commercial posture instead.
Should I start a new chat for each matter?
Yes, and ideally for each task. Long conversations carry over jurisdictions, personas and facts from earlier prompts, which Justia calls being 'haunted by the ghosts of past prompts'; the Singapore Academy of Law guide lists 'start a new chat for each task' as its first rule. Put standing context, such as your playbook and house style, into a Project or custom GPT with written instructions, and keep each matter's chat separate.