Predictive Coding Tipping Point?

February 17, 2012
Matt Miller

Matt Miller

Matt Miller partners with clients to identify, design and implement innovative discovery solutions that often incorporate analytic tools, statistical sampling and validation, and other technology based processes and methodologies.

The Use of Predictive Coding

During the past several years, discussions and commentary pertaining to predictive coding have proven more theoretical than practical. Estimates suggest that companies and law firms are utilizing predictive coding technology on less than five percent of their document reviews (based on our experience, five percent may be generous).

Predictive Coding and DiscoverReady

In his last blog on predictive coding, DiscoverReady CEO Jim Wagner asserted that 2012 might well be the year that predictive coding (or automated review or computer-assisted review or technology-assisted review or whatever you care to call it) finally gains a foothold in the document review space. Based on a recent hearing in front of Judge Andrew Peck in the S.D.N.Y, it looks like Jim’s prediction is right on target.

Judge Peck, E-Discovery Judges in Charlotte

First a little background. During the past two months, the team here at DiscoverReady has been fortunate to interact with Judge Peck on several occasions. In January, Judge Peck travelled to Charlotte to participate in a panel discussion aptly titled “E-Discovery Judges in Charlotte.” The discussion was structured as a mock 26(f) conference/Rule 16 hearing in which Judge Peck joined fellow magistrates Grimm (D.Md.), Facciola ( D.D.C.), Keasler (W.D.N.C), and Hodges (D.S.C.) in providing commentary on e-discovery based on hypothetical cases enacted by representatives from Charlotte corporate heavyweights Bank of America, TIAA Cref, Duke Energy, and Rack Room Shoes, e-discovery counsel from leading law firms, and e-discovery consultants. During this wide-ranging program, Judge Peck echoed his frequently cited sentiments — most recently expressed in “Search, Forward”— that at least certain members of the judiciary would favorably view a discovery process that utilized a documented, defensible predictive coding methodology.

We next ran into Judge Peck at LegalTech, where, during another panel discussion, he again promoted the potential efficacy of computer-assisted review while questioning the efficacy of keyword-search-based discovery methodologies. 

Judge Peck and Da Silva Moore v Publicis Group, et al

Against that backdrop, Judge Peck took the leap that many of the participants in the two panels have been anxiously anticipating. As first reported by LTN, during a February 8 discovery conference in the Da Silva Moore v Publicis Group, et al., matter, Judge Peck entertained a joint discovery proposal wherein the parties proposed the use of predictive coding technology.  Amid the subsequent buzz, many have erroneously suggested that Judge Peck “ordered” the parties to adopt a protocol for e-discovery  that includes the use of Recommind’s predictive coding technology. To the contrary, Judge Peck actually approved a protocol where the parties proposed utilizing an automated solution but differed over the specific manner in which it would be implemented.

Da Silva Moore v Publicis Group Hearing Transcript

The transcript of the hearing provides several important considerations for parties who are considering incorporating automated technology into their discovery plan:

  • First, technologists can and should play a critical role in developing and validating the proposed automated strategy. In this case, technologists, rather than the counsel of record, were primarily responsible for explaining the proposed disputes and solutions to Judge Peck
  • Second, these technologists have to be able to convey complex subject matter (sampling, precision, recall, confidence score) in an easily understood manner. Judge Peck likely has as much, if not more knowledge, regarding automated review than any of his peers, yet on several occasions he had to ask the technologists to “dumb it down”
  • Third, automated review always will be dependent on the decisions that humans make with respect to the documents. As the parties noted and Judge Peck agreed, a “system is only as good as the training that it gets”
  • Fourth, like manual review, there still will be errors associated with automated reviews.  As Judge Peck noted, automated review technologies “certainly work better than most of the alternatives, if not all of the alternatives. So the idea is not to make this perfect; it’s not going to be perfect. The idea is to make it significantly better than the alternative without nearly as much cost.”
  • Finally, as we have suggested, the age of automated review may change the way parties approach and conduct discovery. In this instance, Judge Peck expressly noted that this was not a “black box” solution, since the plaintiffs were “going to see all the documents used to train the system” so they would have complete knowledge as to how the system was trained to find relevance. 

Here at DiscoverReady, we’ve spent years thinking about these and other issues, advising clients on how to conduct an effective automated review process using our proprietary i-Decision process or third-party technologies. We join the rest of the e-discovery world in following this case closely, particularly if we reach the long-awaited holy grail of a formal order approving the use of automated review technology, specifically predictive coding, to conduct discovery.

In the interim, there are many other interesting issues addressed in the full transcript. So take a look while we wait to see how this turns out. It may just be a glimpse into the future.

Back to eDiscovery Blog