Court approval of predictive coding? Not quite, but close.October 20th, 2011
A few weeks ago, our co-founder and CEO Jim Wagner wrote about the status of predictive coding and automated document review in the courts. The gist of Jim’s post was that litigants and counsel considering the use of these technologies for document review shouldn’t wait for a court decision to bless them first. Rather, they should turn to the well-established standard of reasonableness set out in the Federal Rules of Civil Procedure, and ensure that the review — regardless of the particular technology or method used — meets certain key criteria of reasonableness and defensibility:
- Competent professionals executed the review
- Reasonable steps were taken to ensure the quality of the process
- The process was well-documented
- The process was transparent, such that the basis for any automated decision can be readily understood and explained
- The parties conducted adequate sampling to ensure accuracy rates as well as appropriate precision and recall
In an apparent case of great minds thinking alike, Magistrate Judge Peck of the Southern District recently echoed Jim’s thoughts in a Law Technology News article.
Before addressing the reasonableness and defensibility of automated review technologies, Judge Peck first skewers the notion that human review of documents is the “gold standard” of document review process. It should not be, in light of studies that “clearly demonstrate that computerized searches are at least as accurate, if not more so, than manual review.” He also examines the well-accepted use of keyword searches in connection with human review, noting that keyword searching often is fraught with problems and typically generates very low precision and recall.
Judge Peck also shares Jim’s observation about precedent, remarking that “[t]o my knowledge, no reported case (federal or state) has ruled on the use of computer-assisted coding. While anecdotally it appears that some lawyers are using predictive coding technology, it also appears that many lawyers (and their clients) are waiting for a judicial decision approving of computer-assisted review.”
While acknowledging that such a decision may be a long time coming, Judge Peck offers guidance to litigants interested in using these technologies now. From his judicial perspective, if faced with a challenge to the use of predictive or automated coding, he will want to know “what was done and why that produced defensible results.” He would expect to see statistics on the technology’s precision and recall. “Proof of a valid ‘process,’ including quality control testing,” will also be important in establishing defensibility, he writes.
Judge Peck concludes his article by saying this: “Until there is a judicial opinion approving (or even critiquing) the use of predictive coding, counsel will just have to rely on this article as a sign of judicial approval. In my opinion, computer-assisted coding should be used in those cases where it will help ‘secure the just, speedy, and inexpensive’ (Fed. R. Civ. P. 1) determination of cases in our e-discovery world.”
Thanks, Judge Peck. We couldn’t have said it better.