Document Type

Article

Publication Date

Winter 2006

Keywords

Law school rankings, Social Science Research Network, SSRN, U.S. News & World Report, Empirical legal studies, Bernard S. Black, Paul L. Caron

Disciplines

Applied Statistics | Legal Education | Legal Writing and Research

Abstract

One noteworthy feature of the SSRN-based rankings is the high correlation between them and other rankings. Black and Caron report correlation coefficients between their two Social Science Research Network (SSRN) school rankings (one based on downloads from SSRN and one based on the number of papers posted on SSRN) and six other published rankings. The correlations provide a useful and creative measure of consistency across studies. If ranking studies are highly correlated, then the least expensive and most efficient study to conduct can be used without incurring the expense and delay of the more labor-intensive ranking methods. SSRN has a substantial database and software that generates its rankings. SSRN-based rankings' correlations with other rankings raise the possibility that labor-intensive measures, such as those taken by U.S. News & World Report ("U.S. News"), Eisenberg and Wells, and others, can be replaced by SSRN's more automated methodology.

This Commentary first suggests one way in which the SSRN correlations with other ranking studies are more limited than they first appear. SSRN correlations with other studies decrease in samples limited to schools that most directly compete—for example, in samples of schools ranked in the top 10 to 20 as measured by SSRN downloads. Under some reasonable approaches, no statistically significant correlations exist between SSRN downloads and other ranking systems. This is not to suggest that SSRN downloads provide little useful ranking information. That they provide some information about so many schools, without the need for labor-intensive methods, is important. They have the ability to identify schools that might otherwise appear in no ranking system. And, like any objective ranking system, they offer a method to rank all schools without depending on subjective opinions. But the use of SSRN downloads to distinguish among peer law schools—schools that in fact compete with one another—should not be premised on their generally high correlations with other ranking studies across the full range of law schools.

Professors Black and Caron do not claim that SSRN rankings should replace the more labor-intensive methods. My point is simply a caution for consumers of the rankings to use the rankings for purposes for which they are most appropriate. By adding a new set of objective metrics, SSRN should enable consumers to have more information than would otherwise be available. No one metric is so clearly the best such that ignoring others seems appropriate.

I then suggest one reason why SSRN-based rankings may be at a disadvantage compared to other objective ranking systems, particularly when one seeks to make fine distinctions among peer institutions. The SSRN database may provide less comprehensive coverage of downloads than competing systems, such as Westlaw and LexisNexis, provide about productivity and citations.

Finally, I suggest that SSRN's database offers the possibility to raise the standard of statistical analysis of ranking studies. It should take only limited additional programming to produce not just ordinal rankings but also other useful statistical outputs, such as meaningful clusters of schools.

Publication Citation

Published in: Indiana Law Journal, vol. 81, no. 1 (Winter 2006).

Download

Share

COinS