Hey Claude, Evaluate This!

Last fall, ChatGPT took the world by storm, allowing millions to experiment with “generative artificial intelligence” through a simple and intuitive interface. ChatGPT’s success drew competitors, one of which is called Claude, whose creators claim it can analyze long PDF documents. 

To see what Claude could do, I uploaded the petitioners’ and respondents’ briefs from Dobbs v. Jackson Women’s Health, the 2022 case that overruled Roe v. Wade. The surprisingly high quality of the results has significant implications for the review of complex legal documents.