Legal practice should use tools that learn from you.
I was chatting this morning with a Personal Injury attorney who had recently started using a firm-deployed legal AI product and was delighted with its ability to build a medical record chronology and interpret physical therapists' notes.
I had not tried out his new tool, but I have spent time comparing Claude CoWork, Claude legal skills, Westlaw’s CoCounsel*, Everlaw’s Deep Dive and Writing Assistant, Gemini, CoPilot, and others. I have found myself disappointed with the “purpose-built” legal tools. Our conversation got me thinking about repeatability and improvement, two features that are under-appreciated among new adopters of AI products.
I note at the outset that as an industry, we have been appropriately cautious in balancing functionality against valid concerns for the security of client information, privileged communications, and work product.But tools marketed to attorneys should and do have significant security protections in place to avoid client information being used to train a public model, the same way cloud-based eDiscovery has addressed security concerns and is now the industry standard.
Say that I hand a non-learning AI like CoCounsel or Everlaw Writing Assistant some emails and a police report, and tell it to draft a demand letter. I read the draft, give feedback, and request specific changes — to fix the formatting, the structure, the language use, or the tone. Will it remember for my next demand letter?
Answer: No. In legal-specific tools that lack persistent memory, at best, you can write a master prompt articulating your preferences and store it for future uses. In tools with persistent memory, you can instruct it to save those preferences once and apply them always.
Some questions you should consider asking when adopting a new AI solution:
Who defines the output? If “build a medical chronology” is a pre-loaded skill built by the platform, can user feedback be incorporated? I found that in CoCounsel, it disregarded formatting and structure instructions in my prompt because I had called upon its “generate questions” skill, which determined the output.
Does it empower you to check its work? I often reflect on this question in considering Everlaw’s AI coding tools vs. RelativityAIR for Review — Relativity has defined the defensible workflow and walks you through it, while Everlaw is not prescriptive and may empower overconfidence. Does the AI platform have a spot-checking mechanism built in, such as for medical records? Is it easily linking you its sources and cites, and giving you guidance on where it drew some iffy inferences? In a structured request for feedback, the AI should be building its understanding of the case, your preferences, and the underlying documents. You want to know whether corrections from one session inform the next one, and how. What will happen when the task falls to your least experienced associate?
What makes you special as an attorney is your judgment and experience. Your approach to letters, memos, motions, memory aids, and chronologies are extensions of that experience.
Some of the purpose-built legal platforms have to relearn your preferences and tone on every. single. task. That truly limits their usefulness, because they are forever writing their creators' demand letters, rather than yours.