ABSTRACT: Scholars have increasingly investigated “crowdsourcing” as an alternative to expert-based judgment or purely data-driven approaches to predicting the future. Under certain conditions, scholars have found that crowd-sourcing can outperform these other approaches. However, despite interest in the topic and a series of successful use cases, relatively few studies have applied empirical model thinking to evaluate the accuracy and robustness of crowdsourcing in real-world contexts. In this paper, we offer three novel contributions. First, we explore a dataset of over 600,000 predictions from over 7,000 participants in a multi-year tournament to predict the decisions of the Supreme Court of the United States. Second, we develop a comprehensive crowd construction framework that allows for the formal description and application of crowdsourcing to real-world data. Third, we apply this framework to our data to construct more than 275,000 crowd models. We find that in out-of-sample historical simulations, crowdsourcing robustly outperforms the commonly-accepted null model, yielding the highest-known performance for this context at 80.8% case level accuracy. To our knowledge, this dataset and analysis represent one of the largest explorations of recurring human prediction to date, and our results provide additional empirical support for the use of crowdsourcing as a prediction method. (via SSRN)
We have a paper on Abnormal Returns and Supreme Court decision making – this looks like a prime candidate for casino stocks, etc. (not a guarantee but a real possibility) — if only we had the technology to predict Supreme Court cases using methods such as crowds and algorithms 🙂
Fish & Richardson is one of the largest IP firms in the US so it is cool to see them exploring these ideas. If you look at this intro using Microsoft Azure – this is very on point with lots of we have been saying about the mix of semistructured data and #MLaaS (machine learning as a service) … and why we teach both an introduction to quant methods and a machine learning for lawyers course.
I was revisiting some of our old stuff for this Oslo event -early on for us on our #LegalPhysics #LegalAnalytics path – published in Physica A – “By applying our sink clustering method, we obtain a dendrogram of the network’s largest weakly connected component shown in Fig. 4. However, despite their general topical relatedness, these two clusters of cases engage substantively different sub-questions, and are thus appropriately divided into separate clusters. While not a major focus of the docket of the modern court, the early court elaborated a number of important legal concepts through the lens of these admiralty decisions. For example, the red group of cases engages questions of presidential power and the laws of war, as well as general interpretations of the Prize Acts of 1812. Meanwhile, the blue cluster engages questions surrounding tort liability, jurisdiction, and the burden of proof.”
I do not speak at many (any) con law themed events but I am happy to be part of this conversation as it is related to making con law a more scientifically inclined field of human endeavor. #Science #LegalScience #Hashtag
ABSTRACT: In this paper, we investigate the application of text classication methods to support law professionals. We present several experiments applying machine learning techniques to predict with high accuracy the ruling of the French Supreme Court and the law area to which a case belongs to. We also investigate the inuence of the time period in which a ruling was made on the form of the case description and the extent to which we need to mask information in a full case ruling to automatically obtain training and test data that resembles case descriptions. We developed a mean probability ensemble system combining the output of multiple SVM classiers. We report results of 98% average F1 score in predicting a case ruling, 96% F1 score for predicting the law area of a case, and 87.07% F1 score on estimating the date of a ruling
Yesterday was the 2nd Edition of The Forum on Legal Evolution – Hosted at Northwestern University Pritzker School of Law — The Forum is comprised of legal innovators and early adopters, organized around a shared interest in the changing legal market. Paul Lippe + Mark Chandler received lifetime achievement awards. Thanks to William Henderson and his team for organizing the event!
Today – on behalf of my co-authors — I presented at the University of Chicago – Workshop on Judicial Behavior – Organized by Lee Epstein, Frank Easterbrook, Dennis Hutchinson, William Landes and Richard Posner. I think we have a very appropriate UChicago styled paper on Judicial Decision Making and Stock Market Movements.