Home -> Law Blog Directory -> Academic Blogs -> Legal Theory Blog

Academic

: Legal Theory Blog

Heise and Eisenberg on Evaluation of Empirical Legal Studies

By Larry Solum

Michael Heise and Theodore Eisenberg (Cornell University and Cornell University) have posted <a href="http://www.google.co.uk/intl/en/landing/translateforanimals/">Empirical Evaluation of the Impact of Empirical Legal Studies on Judicial Behavior</a> on SSRN.  Here is the abstract:<ul> Empirical legal scholars assume that sound empirical work provides inputs (information about the causal effect of various legal rules( that will have a positive effect on outputs (judicial decisions and opinions that establish rules).  Until recently, this assumption was based entirely on anecdotal evidence--usually in the form of a report that a judicial opinion reached a better outcome by relying on a particular empirical study.  Recently, we have come to doubt the conclusions reached on the basis of such anecdotal evidence.  In Exxon Shipping Co. v. Baker, the Supreme Court relied in part on an article by the present authors and others to state that empirical studies disprove the existence of runaway punitive awards, but the Court also drew conclusions about the unpredictability of damage awards that we have shown, in a prior article ("Variability in Punitive Damages: An Empirical Assessment of the U.S. Supreme Court's Decision in Exxon Shipping Co. v. Baker") were actually contradicted by by our data.</ul> <ul> The current investigation is based on a sample of 213 opinions issued by the federal courts that rely on empirical work as the basis for a significant element of the decision.  This sample was culled from a much larger set of decisions rendered by the federal courts between January 1, 2005, and December 31, 2009.  Each decision was independently evaluated by two experts in empirical legal studies to determine whether the factual conclusion stated in the opinion was in fact supported by cited work.  Our analysis of the resulting data reveals a depressing conclusion.  Each expert was asked to rate the support relationship on a 10 point scale, with 1 representing "Conclusion stated in opinion bore no relationship to cited empirical study" and 10 representing "Conclusion was fully supported by the cited study and fully acknowledged the limitations contained in the cited study."  The average score of the 213 opinions was 1.3.  The maximum score was 4, and this score was achieved by only 1 opinion.</ul> <ul> Based on these results, we tentatively suggest that the implementation of two recommendations.  First, the Judicial Council of the United States should immediately begin to plan an implement a rigorous program of training for federal judges and law clerks on the evaluation and use of empirical legal studies.  Second, pending the implementation of this program, the production of empirical legal scholarship should cease immediately.  The Journal of Empirical Legal Studies should immediately cease publication, and the Conference on Empirical Legal Studies should postpone future meetings until the training program has been fully implemented.</ul>

Full post as published by Legal Theory Blog on April 02, 2010 (boomark / email).