Gurpreet S. Bal follows litigation closely — not because he is a litigator, but because his transactional clients end up in disputes, and the quality of legal representation in those disputes matters. His read on AI in litigation is blunt: "Mata v. Avianca should be required reading for every lawyer in practice today. It is not a story about technology failing. It is a story about a lawyer who did not do their job." The hallucination problem has not been solved. Courts are watching, and the consequences of getting it wrong are serious.
Bal brings a transactional lawyer's eye to litigation AI risk — focused on outcomes, accountability, and what clients actually need to know.
In 2023, a federal judge in the Southern District of New York sanctioned attorneys who submitted a brief citing six cases that did not exist, generated by ChatGPT and filed without verification against any legal database. The lesson is not that AI hallucinated — that is a known property of the technology — but that a lawyer submitted fabricated citations to a federal court because they trusted an AI tool to do a job requiring human verification. Model Rule 3.3 prohibits knowingly making false statements of law to a tribunal, and ignorance of the technology is not a defense.
In 2023, a federal judge in the Southern District of New York sanctioned attorneys who submitted a brief containing citations to six cases that did not exist. The lawyers had used ChatGPT to research case law and submitted the AI's output without verifying the citations against any legal database. When opposing counsel pointed out the citations were fictitious, the lawyers asked ChatGPT to confirm they were real — and it did. The court imposed sanctions, ordered the lawyers to notify the judges falsely cited in the fabricated opinions, and issued a ruling that became a landmark in AI-related professional misconduct. The critical point is not that AI hallucinated. Large language models hallucinate. That is a known, documented property of the technology. The critical point is that a lawyer submitted fabricated citations to a federal court because they trusted an AI tool to do a job that required human verification. Model Rule 3.3 prohibits knowingly making false statements of law to a tribunal. Ignorance of the technology is not a defense.
Legal-specific tools using retrieval-augmented generation ground their outputs in a verified legal database rather than generating citations from training data, making them meaningfully better than general-purpose LLMs — but not hallucination-free. Any system that synthesizes information and produces natural language has some probability of error; the difference is order of magnitude, not kind. A lawyer using Harvey, Lexis AI, or Westlaw AI is in better shape, but the obligation to verify cited authority before submitting it to a tribunal is unchanged. The tool does not carry the bar card; the lawyer does.
The legal tech industry's response to Mata was to build legal-specific AI tools with retrieval-augmented generation — systems that ground their outputs in a verified legal database rather than generating citations from training data. Those tools are meaningfully better than general-purpose LLMs for legal research. They are not hallucination-free. Any system that synthesizes information and produces natural language output has some probability of error. The difference is order of magnitude, not kind. A lawyer using Harvey, Lexis AI, or Westlaw AI for case research is in materially better shape than one using a general-purpose chatbot — but the professional obligation to verify cited authority before submitting it to a tribunal is unchanged. The tool does not carry the bar card. The lawyer does. Courts increasingly require attorneys to certify that AI-generated filings have been reviewed for accuracy. That certification is not a formality. It is a representation to the court.
AI-generated audio, video, and images have become realistic enough that authentication standards developed for traditional media are straining. The Federal Rules of Evidence require a proponent to show evidence is what it purports to be, and courts have historically accepted contextual authentication — voice identification, metadata, corroborating circumstances. Deepfakes can satisfy all of those markers while being entirely fabricated. The framework is still developing and forensic detection tools are in an arms race with generation tools, so litigators must scrutinize digital media evidence and be prepared to challenge authentication on both sides.
The other end of the courtroom AI problem is evidentiary rather than research-related. AI-generated audio, video, and images have become sufficiently realistic that authentication standards developed for traditional media are straining under the weight of the technology. The Federal Rules of Evidence require that evidence be authenticated — that the proponent demonstrate it is what it purports to be. For video or audio evidence, courts have historically accepted contextual authentication: this is a recording of X because Y identified the voice, because the metadata shows it was captured on Z device, because the circumstances corroborate it. AI-generated deepfakes can satisfy all of those contextual markers while being entirely fabricated. Several courts have grappled with deepfake evidence challenges. The framework is still developing, and forensic AI-detection tools are in an ongoing arms race with generation tools. Litigators need to treat digital media evidence with a level of scrutiny that was not necessary five years ago — and to be prepared to challenge authentication on both sides.
Clearly appropriate, well-established uses include large-scale document review and privilege logging, first-pass research synthesis, deposition preparation, and contract analysis in commercial disputes — tasks where AI accelerates a human-supervised process. Clearly off-limits today are autonomous filing of court documents without attorney review, AI-generated content used as original evidence, and reliance on AI research without independent verification. The middle ground, such as AI-assisted brief drafting, requires careful human oversight. The safe principle: AI as the accelerant, lawyer as the author.
The uses of AI in litigation that are clearly appropriate and already well-established include large-scale document review and privilege logging, first-pass research synthesis, deposition preparation (identifying prior statements, flagging inconsistencies in large document sets), and contract analysis in the context of commercial disputes. These are tasks where AI accelerates a human-supervised process and where errors are caught by the supervising attorney before they cause harm. The uses that are clearly not appropriate today include autonomous filing of court documents without attorney review, any use of AI-generated content as original evidence, and reliance on AI legal research without independent verification. The middle ground — AI drafting of briefs, AI-assisted motion practice, AI-generated expert report summaries — requires careful human oversight and is evolving rapidly. The safe principle: AI as the accelerant, lawyer as the author. The moment those roles reverse, you have a professional responsibility problem.
Federal and state courts have begun requiring disclosure of AI use in filings, with several districts requiring lawyers to certify that AI-generated content has been reviewed and verified, and some requiring disclosure of which tools were used. These requirements are not uniform — they vary by court, judge, and case — but the trend toward mandatory AI disclosure is clear. The body of law is developing quickly and unevenly, and litigators who assume their existing practices are sufficient are taking a risk their clients are not well-positioned to absorb.
Federal and state courts have begun requiring disclosure of AI use in filings. Several districts require that lawyers certify that AI-generated content has been reviewed and verified. Some courts have gone further, requiring disclosure of which tools were used. These disclosure requirements are not uniform — they vary by court, by judge, and by case — but the trend toward mandatory AI disclosure is clear. Lawyers who treat AI as a silent tool that requires no disclosure are increasingly on the wrong side of developing practice norms. Beyond disclosure, courts are thinking carefully about the evidentiary, procedural, and professional responsibility questions that AI raises. The body of law is developing quickly and unevenly. Litigators who are not tracking it — who assume their existing practices are sufficient — are taking a risk that their clients are not well-positioned to absorb.
Gurpreet S. Bal is a corporate partner with 16 years advising on private equity, merger transactions, and public offerings for companies and investors at three of the world's top law firms. He has represented clients in hundreds of transactions with aggregate deal value exceeding $60 billion across AI, semiconductors, fintech, and emerging technology. For more information and to get in touch, visit gurpreetbal.com.