As an appellate lawyer, I’m in the writing business. About a year ago, I started learning about artificial intelligence technology that could help me and my colleagues write better and faster.
And since ChatGPT came online, I’ve tried to do a little bit of everything with it to learn what it can do that I can build into my law practice. In this article I explain why I now think of it as my “Pocket B.S. Detector.”
I’ve talked to ChatGPT about serious legal questions (“Can parties create a partnership by conduct if they have agreed in writing to not be partners?”) and whimsical ones (“Please write a limerick about the Cowboys-49ers game.” I even interviewed it in mid-January for my Coale Mind podcast, about how it saw its own future role in our legal system.
From these talks I’ve learned that ChatGPT is not much for details. In our interview, for example, ChatGPT was wrong about when, and in what case, the Supreme Court overruled Roe v. Wade. And because ChatGPT’s systematic review of historic data largely ended when it went online last November, it has little to say about current events and issues in today’s headlines.
What ChatGPT can do, astonishingly well, is identify and discuss a handful of key points on just about any issue you can think of. “Please write an essay with five reasons why blue is the best color.” “Please discuss the pros and cons of the parol evidence rule.” I really think that it can write five organized paragraphs on any topic that humans have addressed in publicly available writings. And ChatGPT produces its work product in, at most, a few seconds.
While ChatGPT’s “gee whiz” abilities are fun, I don’t really need more than one sonnet about how traffic signals work. I’ve focused on its remarkable short-essay ability to find a role for ChatGPT in my daily professional life as a “Portable B.S. Detector.”
Here’s what I mean by that. As an advocate for a client, my team and I have to accept and share the same core beliefs about the client’s position.
But over time, the positive force created by those shared beliefs can inadvertently turn into the negative phenomenon of “groupthink,” where good ideas aren’t fully considered because they challenge a basic premise of our position. Our advocacy then risks becoming less effective by having not fully considering a thoughtful challenge to our position.
Groupthink is a constant occupational challenge for professional advocates. Humans are social animals and naturally want to get along with those around you.
ChatGPT provides a quick antidote, because it doesn’t care about anyone’s feelings. Its only “desire” is to execute its programming by answering the questions put to it.
If you ask ChatGPT about the pros and cons of a particular argument, its answer may be wrong on key facts or not fully informed about recent news. But you can be confident that it will be delivered without any bias towards what you want to hear.
Of course, I still try to stump ChatGPT with off-the-wall requests (“Do you know who wrote The Book of Love?”). But more seriously, I now try to maintain a habit of asking it basic B.S. detection questions, asking it for arguments against a position I want to take in a case or reactions to a potential case theme.
Nine out of 10 times it doesn’t tell me anything that I didn’t already know. But if I gain a useful insight only 10 percent of the time and by doing so keep myself from falling in the trap of groupthink, that’s an enormous, potentially case-winning benefit — obtained for free with a minimal time investment.
That’s why I have a positive view about what ChatGPT will bring to the legal profession and the court system. If it subtly prods people to test their assumptions over time, those reminders should lead to better-reasoned assumptions and thus higher-quality advocacy. I’m excited to see what’s coming next — and to see if I can finally come up with a goofy enough request to genuinely stump ChatGPT.