Transcript:

Intro: Hello, and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day. 

Anthony: Hello, this is Anthony Diana from Reed Smith, and welcome to Tech Law Talks. Today, we are continuing our podcast series on AI-enabled e-discovery. The podcast series will focus on practical and legal issues when considering using AI-enabled e-discovery with a focus on actual use cases. So joining me today are Therese Craparo and Samantha Walsh from the New York office of Reed Smith. Welcome, guys. So now let's jump in and get started. Today, we're going to be talking about privilege and the use cases around using AI-enabled e-discovery for privileged issues. So let's just start with the problem that everyone has, which is privilege reviews and privilege logging represent one of the highest costs of e-discovery. And it has been a challenge, I think, you know, over the past, you know, 20 years, as everything has gotten better in terms of operationalizing and squeezing it, squeezing out as much efficiency as possible, privilege continues to be a challenge that, frankly, hasn't really been solved. So the question is, can GenAI or other types of artificial intelligence, can it be effectively used to start bringing down these costs? 

Samantha: Yeah, I mean, I agree. Dealing with privilege is a huge challenge and a huge cost because it's so important to get it right. And getting it right is a pretty heavy lift. So we've definitely seen success using AI for handling privilege in e-discovery. I think in particular with privilege review and QC of privilege review, but also just in generally identification of potentially privileged material and in generation of privilege logs, which I think is one of the more resource intensive aspects of the privilege. privilege, endeavor. 

Anthony: And so, Therese, how do we see AI currently being used in the privilege review process? And how does that differ from sort of traditional manual review? 

Therese: The main area where we've seen a lot of success with the use of AI with privilege review has been with the technology-assisted review models in sort of directing and focusing privilege review. And really, you know, it's like any kind of TAR review, it's augmenting human review, right, and helping to make the human review more efficient and more effective. So just in terms of talking about it in real terms, we had a case, we had hundreds of thousands of privileged documents, tens of millions of documents to be reviewed. So really a massive case. And as Sam already highlighted, right, identifying privilege is critical, right? You have to be precise, you have to find the information. There was a lot of concern with the speed with which the review needed to be done because, of course, we're always under the gun and under deadlines to produce documents of getting it done, but also getting it done precisely and making sure that privileged documents weren't produced and other types of confidential information wasn't produced. And so effectively, what we did is use a TAR model, train the model. What we had already known were privileged documents so that we could score the privileged, likely-to-be-privileged documents in the process. In doing that, what we elected to do to streamline the review was anything that had a certain high score, so after reviewing it and QC-ing the model, highly likely-to-be-privileged got put into a privilege-slash-responsive review queue, like direct into the queue, so that we weren't going through several levels of review where that may not be necessary. Highly likely-to-be-privileged goes to the privileged review. But then even with the reviewers who are doing the review of the remaining documents, they could still see the score of the documents, if it was likely to be privileged, which gave them a little bit of insight, maybe pay a little bit more attention here because this may be a privileged document, in order just to help everyone focus on things likely to be privileged. And then on the flip side, for the QC purposes, use the model to say, is there anything in the production sets that have a higher score as privilege? So did we miss anything? Did something slip through the cracks? Can we take a look at that just to make sure that we were doing that extra check, not just relying on search terms and things like that to try to find the information? So it's not like it made the review magically, you know, done with privilege, but it made it much more efficient, much more effective. The time that went into it to get the model ready saved so much time on the back end in terms of the ability to, you know, get the documents through the door, out in the production, but feeling confident that we had been able to identify the privilege documents. And so I think that there's a real, you know, live use case really was effective. And I think it's something people really should be thinking about. Even where people are still, for reasons that escape me, nervous about using TAR in review, you know, using it to identify privilege for things that are going through the review to make your review better and to help people to identify privilege information, it really just seems like a no-brainer to me. 

Anthony: And I think, again, I think people get, and we've heard this from lots of clients and outside counsel, whatever, they're very nervous about using machine learning TAR in this, and it hasn't been used that much. I mean, the reality is that very, very, very few people and organizations are actually using even something like TAR, which has been around forever. 

Therese: Anthony, you mean not everyone is as enthusiastic about AI and e-discovery as you are? 

Anthony: It should be done. It should be done. We used it. But again, I think a few concepts here, and I think this is important, Therese, as you mentioned, is the way we used it wasn't, oh, AI made the privilege call, right? It's a tool. And I think we found it effective, right? What you said is it's one, establishing workflow, right? Which is we're scoring it. And people don't understand it's scored. So it could be a nine or whatever, and it's scored. And nine means that it's very likely to be privileged and zero is, you know, it's not going to be privileged at all. And obviously, you know, you can develop workflows based on that. And I think a lot of people that have used it have tried to use this. And we're going to be seeing that also, I think, with GenAI. So that's that concept of scoring the document on whether it's privileged. But again, if you're just using it to, you know, get a workflow, so you have people that are really focused on privilege, looking at that is helpful. And then, like you said, the QC, I mean, there's two areas where there's not a lot of risk associated with that. It's a tool that helps, but you're not saying, oh, I relied on AI to determine privilege. No, it's just helping your teams, whether it's workflow or QC, that to me is very low risk, which I agree with is why people should be using it. So Sam, in terms of, we talked a little bit about that example, which is helpful, which is historically, but. Sort of in the future, as we start thinking about what types of AI models or approaches are sort of, you know, going to be commonly used to identify privileged content, right? Is it, we talked about rules-based, machine learning, GenAI, what are your thoughts on, you know, sort of what the approaches are going to be? 

Samantha: Yeah, well, I mean, as we've discussed, machine learning is pretty tried and true in e-discovery and, you know, it does work quite well in, you know, when trained properly in identifying, you know, things that are likely to be privileged. I think what is sort of interesting or exciting about GenAI in this area is that instead of maybe providing a score, a GenAI can provide a rationale, right? This document is privileged because this. And, you know, I think that just sort of advances the, the ball a little bit in terms of what we're getting out of AI for privilege, possibly reducing the amount of human effort required in terms of training. When you're using TAR, there's still a fair amount of human review that has to happen. And ultimately, after the model runs through the data set, you still have to review everything. So I think the potential to maybe, you know, lighten the load even more and a little bit maybe more reliably is there with GenAI. 

Therese: Yeah. And I also think, and I think this is one of the areas, a little bit of ECA, but also for privilege purposes, one of the challenges with privilege is you don't know what you don't know. Right? So you don't know what privileged communications were happening necessarily. You don't know what topics were out there. And the number of times we've done a review, you've done everything you can, you give people all the information you can give them to what to look for for privilege, and then something gets produced and you're like, we didn't know that communication was happening and that should have been on our privilege list, right? I think one of the other promising things about GenAI is that it's interrogating your data early on to say, what should be privileged? What kinds of communications are out there? What kind of topics are out there? How do I use this to surface to better prepare? Like Sam said, better prepare if you're going to use TAR, better train the model because I found the things ahead of time that it needs to know. And I think that that's something that TAR can't do, right? Machine learning doesn't do that for you because you have to tell it what to look for. GenAI can find things for you. If you're asking the right prompts, if you're doing the right interrogation of the data, it can surface things for you that you don't even know about and then help you to better prepare for your privilege reviews. And I think that's an area that just really could make a material difference in helping with making sure that privilege reviews themselves have all the information they need to find the privileged information as you go through the review. 

Anthony: Yeah. And I know like some of the vendors out there are, are sort of doing a combination of, you know, our search terms, everything, and that's sort of their normal workflow, which is they have already developed a specific sort of identifier, a long language, large language model that is, that is geared for privilege, which is sort of your base. And Therese, your point, And it may get general stuff. And then you layer that and sort of hone the tool using TAR. So if you have a general model that's based on privilege, which could help identify privilege information, again, never rely on that, but it's a nice base. But then the concept of now I'm going to develop a privilege tool that's unique for this business line, this case, this organization, you can use samples, right? Like we do for TAR to train that tool better. so it's even more precise and more relevant. And obviously you would assume that the performance is going to be better. And then there's, I think, in terms of rule-based and stuff like that, you can start thinking about using search terms for when privilege may break, right? So if it's sent to a third party and those types of things, you can sort of model it and put it all together. But that means you sort of have to become an expert on how these tools work and the like, which is definitely part of it. But I think this is something I'm super excited about, because I do think this is an opportunity for us to really move the needle on how people conduct privilege. So, Therese, in terms of, obviously, we just talked about this, is obviously one of the concerns is how good are these tools in actually getting the nuance, right? And we know that's going to be the pushback from outside counsel and everybody is, it's too nuanced for these tools to work. What is the response to that? 

Therese: It depends on how you're using them and the oversight that's provided when you're using them, right? I mean, right now, are the AI tools great at making nuanced evaluations between attorney-client privilege or work product or things like that? No, but you're not relying on them for that. And I think that's the important thing. And I think they'll get better. Frankly, we've seen such incredible advances in AI. I think we'll see that move along pretty quickly. But I think it's important to remember, again, you're not asking the AI to make a decision for you. You are asking the AI to surface things for you to make so that a decision can be made and to find the things that are most likely to fit the criteria, right? And then you can do the review. So we're not asking AI right now to be able to be super precise in the type of privilege. As Sam already mentioned, There are some of these where it's actually helpful because it can give you a reason why it thinks. Not that it thinks. The reason why the algorithm is telling it that it may be privileged. So it can help to give you a background to make a decision. But the real goal of these tools is how do I find the things that are likely to be similar so I can make faster decisions? And they do that very well. So I think that it's always a nuance is that people, if you think you're using AI as a magic bullet that will give you answers and alleviate us from having to do any work, it will not do that. And we don't want it to do that right now, right? We want there to be a human element. Goodness gracious, it's hard when people are looking at documents and have debates over whether or not it should be considered privileged, right? It isn't an easy answer. But I think they are very good at surfacing categories, finding things that look alike, so that you can move, go better, faster, quicker in your reviews. And that's really what we're trying to do here. 

Anthony: Yeah, so, Sam, where do you see the use of AI where it could be helpful in terms of the privilege process? 

Samantha: What I would love to see is fully automated privilege logging. Privilege logs are the worst. They're totally a necessary evil. And they are very expensive and often not even used. I do actually think that generative AI could probably produce a pretty good privilege log. And I think, you know, as far as we've come streamlining the privilege logging process, it's still a slog. And I think to get any kind of meaningful privilege log is a slog. You can give a metadata log pretty easily. And I do think that large language model would probably make pretty quick work of summarizing very briefly, you know, 10,000 privileged documents for your adversary. 

Anthony: Yeah, no, and I think, look, I think there's cost issues associated with it. I think the cost will come down like everything else. I tend to think that this is probably an area where AI is going to dominate once it gets accepted and stuff and the price point comes down because as you said, it costs a ton of money, not much use either, but it's going to be AI tools talking to AI tools, right? You're going to submit your privilege log that was created by an AI tool with QC and the like, and then the other side is going to use their AI tool to analyze it and highlight things or whatever. So it'll be interesting, but I think there's an area where it definitely is going to work as soon as the cost comes down, because I think it's relatively expensive right now, but it is also awful. The other thing that I think is interesting is one of the challenges we have is that documents are tag privilege, but shouldn't be privileged. And then you don't find it until you do the privilege log. It's a nice way if you do it with, you know, if you're doing a GenAI tool, it's going to have a problem describing why it's privileged, right? So you can sort of see it and say, well, it doesn't know why it's privileged, right? So, well, that's maybe one that we should throw back and say it's, we can produce it. So it's almost like a QC of your privilege calls, because if the AI tool can't describe it as to why it's privileged, then maybe it shouldn't be privileged at all. So I do think that's an area where it's certainly ripe for people to start using. It'll be interesting to see if that becomes standard, that we start using these as a standard process if the price point is right, for all the reasons, Sam, that you mentioned. So exciting times. So thank you both. Again, this is on privilege. We may have others on privilege, But we'll be doing more of these podcasts on AI-enabled e-discovery, among various things. Hopefully you enjoyed it. If you have any questions, obviously you can reach out to Reed Smith. Thanks.

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's Emerging Technologies Practice, please email techlawtalks@reedsmith.com. You can find our podcast on Spotify, Apple Podcasts, Google Podcasts, Readsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.