Today was day one of the University of Chicago Law School's Symposium on Developing Best Practices for Legal Analysis. The aim of the Symposium, and a special forthcoming issue of the University of Chicago Law Review, is as follows:
Our Symposium will bring together academics specializing in a variety of legal subjects to develop methodologies and best practices to guide legal research. This discussion will first consider whether legal scholarship has sufficiently rigorous standards for academic researchers to support claims about changes in legal doctrine. Given the strong case for the argument that legal scholarship lacks such rigorous standards, Symposium participants will also debate the proper methodological principles that should inform how scholars should support arguments about constitutions, statutes, court decisions, and administrative regulations. This debate will aim to incorporate methodological insights from a variety of social sciences and to reach consensus with regard to the best practices for academic legal research, writing, and editing.
I won't attempt a comprehensive account just yet, given tomorrow's important and interesting schedule. Instead, I will offer some initial impressions on the presentations of day one and what emerged from them.
The overarching (if implicit) theme of day one was methodological transparency and humility: enhancing methodological rigor by making methodological assumptions and choices explicit, and consistent with substantive aims and commitments. This theme emerged in the initial presentation of the day by University of Chicago professors Will Baude, Adam Chilton, and Anup Malani, the Symposium's organizers, who argued that legal scholars can make more rigorous and more persuasive doctrinal claims (i.e. about what the law is, and whether and how the law is changing) by adopting the tools of systematic reviews (which are common in management and medical research). They set out the following five generally applicable analytic steps: (1) state the question clearly and unambiguously (seems obvious, they note, but I agree that this is actually rarely done); (2) define the sample of cases; (3) explain how the cases were located (e.g. the search terms used); (4) make explicit the relative weighting of cases (not all cases are equal, after all, but scholars should make explicit the criteria by which they assign relative authoritative weights to cases, especially the so-called "leading cases"); and (5) analyze the sample in order to answer the question stated with the appropriate level of modesty dictated by the outcomes of the foregoing steps.
Perhaps unsurprisingly, this proposal generated a good deal of resistance among Symposium participants, many of whom argued that positive and normative claims cannot always (if ever) be neatly compartmentalized. Professor Lawrence Solum observed, moreover, that the authors' own framework had (ironically) failed to make fully explicit its normative commitments and methodological assumptions, a point not lost on the authors. This epistemological difficulty immediately sprang up in the Symposium's next presentation, on the use of qualitative methods (particularly counterfactuals) for law review writing by Professor Katerina Linos & Ph.D. candidate Melissa Carlson. In particular, the attempt to determine custom in international law is beset by the practice of adjudicators who at once create and determine custom in the very act of searching for it. A related challenge emerged out of the presentation by Professors Fred Schauer & Barbara Spellman on "Analogy, Expertise, and Experience," particularly the idea that lawyers often think (analogically) in terms of the relationships between objects (please note that there was much else to this talk beyond this particular point), sometimes consciously, sometimes not.
The third presentation focused on an empirical approach to searching for the common law. Oren Bar-Gill, Omri Ben-Shahar, and Florencia Marotta-Wurgler concentrated on two doctrinal questions: (1) are "terms-in-the-box" conditions contractually enforceable? and (2) are website privacy policies enforceable contracts? The presenters described their empirical approach as establishing a doctrinal baseline (answering both questions in the affirmative), but this approach, as valuable as it is, also highlighted a potential weakness (or perhaps lowered expectations) of adopting social science methodologies as a part of legal analysis. Regarding the strong judicial approval and enforcement of so-called "shrinkwrap" terms, for example, Professor Richard Epstein asked whether the change since the mid-1990s was caused by judicial reasoning (say, that of Judge Frank Easterbrook of the U.S. 7th Circuit Court of Appeals in particular), or by consumers' (including judges' own) increasing familiarity with and social acceptance of such terms? The authors were not only unable to answer this question on the basis of their empirical work, it was evident that they had not turned their minds to the question, which may itself be a symptom of the narrower band of theorizing that sometimes accompanies descriptive empirical work.
The Symposium shifted next to methods of statutory interpretation. Professor Abbe Gluck presented on her fascinating research into the gap between American canons of statutory interpretation and the actual practices of legislative drafting employed by the staff and committee members of the U.S. Congress, who do much (perhaps most) of their drafting work completely and unabashedly unawares of judicial canons of construction (such as "the rule against superfluities" and "the rule of the immediate antecedent"). While some argue that Congressional intent is essentially a black box beyond legal interpretation (counseling in favour of a purely textual analytic approach), Professor Gluck convincingly shows how scholars and judges can interpret the intent and legislative context of Congressional lawmaking by closely examining the structure and practices of Congressional drafting (which is summed up nicely by the title of her talk, "The CBO Canon (Faint-Hearted Formalism and Why Courts Should Learn About How Congress Actually Works)." Professor Gluck's proposals complemented the presentation of Judge Frank Easterbrook on the radical indeterminacy of statutes, which are premised, not on the theory of cooperative private communication, but on the basis of conflicting interests and intentions. While Judge Easterbrook argued that canons of construction will never be algorithmic (also nicely signaled by his paper title, "The Absence of Method in Statutory Interpretation"), Professor Solum argued that we shouldn't expect them to be, as they are not meant to be legal rules, but merely rules of thumb that tend to work, depending on the context in play.
In "Arguing in Good Faith about the Constitution: Ideology, Methodology, and Reflective Equilibrium," Professor Richard Fallon's returned to the theme of methodological transparency by arguing for a good faith approach to constitutional analysis. For Professor Fallon, legal scholars should try to match up their methodological commitments with their substantive commitments. However, Professor Fallon likewise thinks that scholars' methodological commitments can and should change over time and upon reflection. But as they change, scholars must reflect openly and honestly on such changes, what Professor Fallon calls the "reflective equilibrium hypothesis." Professor Fallon refreshingly argues for constitutional law scholars to open themselves to changing their minds and to be equally open about how their minds have changed.
Professors Cass Sunstein & Adrian Vermeule ("The Unbearable Rightness of Auer") next presented on the administrative law principle of agency deference in the context of clear administrative rules, using this issue to discuss the fallacy of declaring an analytic victory on the basis of the direction of a purportedly perverse incentive without investigating the magnitude of the alleged incentive-cum-hazard. For example, they point out that those who argue against deference to administrative agencies' entitlement to clarify their own rules creates an incentive for agencies to create vague rules that they can clarify as they like down the road. But even if true (they claim the opposite, that agencies aim for clarity in order to bind successors), this directional incentive says nothing about its magnitude, which even in the case of the opposite direction of administrative rule clarity they argue would be negligible at most. Professors Sunstein & Vermeule argue that legal culture is replete with this kind of cognitive error (incentive direction absent magnitude), and that a "best practice" would be to root it out of our analyses (or to make the comparison of direction and magnitude explicit).
In "Concepts Before Precepts: The Central Place of Doctrine in Legal Scholarship," Professor Richard Epstein concluded day one of the Symposium by arguing that public lawyers must look to the private (indeed, even Roman) law origins of public law concepts to guide their analyses. He provided many examples, prominent among them takings law. In the Q&A period following his talk, Professor Epstein advised struggling young legal scholars that, whenever they are told that "it depends on context" (a recurring trope in statutory interpretation, and the common law generally), they should ask how and why, which in my view is a deceptively simple but powerful analytical tool.
My overall impression of day one of the Symposium is that its goal of reaching "consensus with regard to the best practices for academic legal research, writing, and editing" is a laudable aim, if a long-term one. Perhaps the most promising mechanism by which such a convergence may be achieved is by legal scholars becoming much more deliberate and open about both their methodological choices and the relationship between those choices and their substantive priors. Such openness would allow for far more constructive criticism and dialogue among legal scholars and, in the medium-to-long term, may generate a set of common standards regarded as "best practices" for legal analysis. In the meantime, it would certainly make legal analysis more accessible, not only among legal scholars, but also to practicing lawyers, lawmakers, and judges.
Watch this space tomorrow for a further discussion of these issues, including Professor Lawrence Solum's timely presentation on "Originalist Methodology" and Judge Richard Posner's talk on "What Judge's Need in the Way of Legal Research."
