Cornell e-Rulemaking Initiative Publications Copyright (c) 2013 Cornell Law Library All rights reserved. http://scholarship.law.cornell.edu/ceri Recent documents in Cornell e-Rulemaking Initiative Publications en-us Sat, 16 Feb 2013 01:47:04 PST 3600 Regulation Room: How the Internet Improves Public Participation in Rulemaking http://scholarship.law.cornell.edu/ceri/13 http://scholarship.law.cornell.edu/ceri/13 Thu, 14 Feb 2013 11:16:42 PST Cornell eRulemaking Initiative (CeRI) designed and operated Regulation Room, a pilot project that provides an online environment for people and groups to learn about, discuss, and react to selected proposed federal rules. The project is a unique collaboration between CeRI academic researchers and the government. The U.S. Department of Transportation (USDOT) was CeRI's first agency partner and chose Regulation Room as its first open government "flagship initiative." USDOT received a White House Open Government Leading Practices Award for its collaboration in the project. CeRI owns, designs, operates, and controls Regulation Room, but works closely with partner agencies to identify suitable "live" rulemakings for the site and to evaluate success after a rule closes.

]]> Jackeline Solivan et al. Rulemaking vs. Democracy: Judging and Nudging Public Participation that Counts http://scholarship.law.cornell.edu/ceri/12 http://scholarship.law.cornell.edu/ceri/12 Thu, 24 Jan 2013 08:43:11 PST This essay considers how open government “magical thinking” around technology has infused efforts to increase public participation in rulemaking. We propose a framework for assessing the value of technology-enabled rulemaking participation and offer specific principles of participation-system design, which are based on conceptual work and practical experience in the Regulation Room project at Cornell University.

An underlying assumption of open government enthusiasts is that more public participation will lead to better government policymaking: If we use technology to give people easier opportunities to participate in public policymaking, they will use these opportunities to participate effectively. However, experience thus far with technology-enabled rulemaking (e-rulemaking) has not confirmed these assumptions. To the extent that new participants have engaged with the process, their engagement predominantly takes the form of mass comment campaigns orchestrated by advocacy groups. The conventional response to this new participation – by agencies and academics alike – has been to regard mass commenting as worse than useless. Recently, though, Nina Mendelson argued for rethinking this response. Exploring the relationship between rulemaking and democratic government, she proposes that agencies should take account of the value preferences expressed in such comments when rulemaking involves value judgments.

Engaging this important argument, we suggest that not all citizens’ preferences about policy outcomes are created equal. We present a typology that captures important differences in information quality and deliberativeness of preference formation. Unlike electoral democracy (in which participation based on any type of preference is valued), the legitimacy of the rulemaking process relies on a formally transparent process of reasoned deliberation. The types of preferences expressed in mass comments may be good enough for electoral democracy but they are not good enough for rulemaking, even when rulemaking is heavily laden with value choices.

This position challenges both the Web 2.0 ethos and the common open-government belief that more public participation, of any kind, is a good thing. At least with respect to rulemaking and similar complex policymaking processes, more public participation is good only if it is the kind of participation that has value in the process. From our experiences on Regulation Room, we argue that design of successful “Rulemaking 2.0” civic engagement systems must involve a purposeful and continuous effort to balance “more” and “better” participation. We offer several specific design principles for striking this balance, perhaps the most important of which is that a democratic government should not actively facilitate public participation that it does not value.

]]> Cynthia R. Farina et al. Achieving the Potential: The Future of Federal e-Rulemaking: A Report to Congress and the President http://scholarship.law.cornell.edu/ceri/11 http://scholarship.law.cornell.edu/ceri/11 Thu, 03 Jan 2013 12:08:24 PST Federal regulations are among the most important and widely used tools for implementing the laws of the land – affecting the food we eat, the air we breathe, the safety of consumer products, the quality of the workplace, the soundness of our financial institutions, the smooth operation of our businesses, and much more. Despite the central role of rulemaking in executing public policy, both regulated entities (especially small businesses) and the general public find it extremely difficult to follow the regulatory process; actively participating in it is even harder.

E-rulemaking is the use of technology (particularly, computers and the World Wide Web) to: (i) help develop proposed rules; (ii) make rulemaking materials broadly available online, along with tools for searching, analyzing, explaining and managing the information they contain; and (iii) enable more effective and diverse public participation. E-rulemaking has transformative potential to increase the comprehensibility, transparency and accountability of the regulatory process. Specifically, e-rulemaking – effectively implemented – can open the rulemaking process to a broader range of participants, offer easier access to rulemaking and implementation materials, facilitate dialogue among interested parties about policy and enforcement, enhance regulatory coordination, and help produce better decisions that lead to more effective, accepted and enforceable rules. If realized, this vision would greatly strengthen civic participation and our democratic form of government.

]]> Committee on the Status and Future of Federal e-Rulemaking (U.S.) RegulationRoom: Field-Testing An Online Public Participation Platform During USA Agency Rulemakings http://scholarship.law.cornell.edu/ceri/10 http://scholarship.law.cornell.edu/ceri/10 Tue, 20 Nov 2012 07:28:34 PST Rulemaking is one of the U.S. government's most important policymaking methods. Although broad transparency and participation rights are part of its legal structure, significant barriers prevent effective engagement by many groups of interested citizens. RegulationRoom, an experimental open-government partnership between academic researchers and government agencies, is a socio-technical participation system that uses multiple methods to alert and effectively engage new voices in rulemaking. Initial results give cause for optimism but also caution that successful use of new technologies to increase participation in complex government policy decisions is more difficult and resource-intensive than many proponents expect.

]]> Cynthia R. Farina et al. Using Natural Language Processing to Improve eRulemaking [Project Highlight] http://scholarship.law.cornell.edu/ceri/9 http://scholarship.law.cornell.edu/ceri/9 Tue, 20 Nov 2012 07:02:25 PST This paper describes in brief Cornell’s interdisciplinary eRulemaking project that was recently funded (December, 2005) by the National Science Foundation.

]]> Claire Cardie et al. Better Inputs for Better Outcomes: Using the Interface to Improve e-Rulemaking http://scholarship.law.cornell.edu/ceri/8 http://scholarship.law.cornell.edu/ceri/8 Mon, 19 Nov 2012 13:26:44 PST We believe that e-rulemaking does indeed have potential to increase both the transparency of, and participation in, regulatory policymaking. We argue in this paper that this potential can be realized only if the public interface at www.regulations.gov is substantially redesigned.

]]> Cynthia R. Farina et al. Active Learning for e-Rulemaking: Public Comment Categorization http://scholarship.law.cornell.edu/ceri/7 http://scholarship.law.cornell.edu/ceri/7 Mon, 19 Nov 2012 12:57:45 PST We address the e-rulemaking problem of reducing the manual labor required to analyze public comment sets. In current and previous work, for example, text categorization techniques have been used to speed up the comment analysis phase of e-rulemaking - by classifying sentences automatically, according to the rule-specific issues [2] or general topics that they address [7, 8]. Manually annotated data, however, is still required to train the supervised inductive learning algorithms that perform the categorization. This paper, therefore, investigates the application of active learning methods for public comment categorization: we develop two new, general-purpose, active learning techniques to selectively sample from the available training data for human labeling when building the sentence-level classiers employed in public comment categorization. Using an e-rulemaking corpus developed for our purposes [2], we compare our methods to the well-known query by committee (QBC) active learning algorithm [5] and to a baseline that randomly selects instances for labeling in each round of active learning. We show that our methods statistically significantly exceed the performance of the random selection active learner and the query by committee (QBC) variation, requiring many fewer training examples to reach the same levels of accuracy on a held-out test set. This provides promising evidence that automated text categorization methods might be used effectively to support public comment analysis.

]]> Stephen Purpura et al. An eRulemaking Corpus: Identifying Substantive Issues in Public Comments http://scholarship.law.cornell.edu/ceri/6 http://scholarship.law.cornell.edu/ceri/6 Mon, 19 Nov 2012 12:14:34 PST We describe the creation of a corpus that supports a real-world hierarchical text categorization task in the domain of electronic rulemaking (eRulemaking). Features of the task and of the eRulemaking domain engender both a non-traditional text categorization corpus and a correspondingly difficult machine learning task. Interannotator agreement results are presented for a group of six annotators. We also briefly describe the results of experiments that apply standard and hierarchical text categorization techniques to the eRulemaking data sets. The corpus is the first in a series of related sentence-level text categorization corpora to be developed in the eRulemaking domain.

]]> Claire Cardie et al. Facilitating Issue Categorization & Analysis in Rulemaking http://scholarship.law.cornell.edu/ceri/5 http://scholarship.law.cornell.edu/ceri/5 Mon, 19 Nov 2012 11:48:17 PST One task common to all notice-and-comment rulemaking is identifying substantive claims and arguments made in the comments by stakeholders and other members of the public. Extracting and summarizing this material may be helpful to internal decisionmaking; to produce the legally required public explanation of the final rule, it is essential. When comments are lengthy or numerous, natural language processing and machine learning techniques can help the rulewriter work more quickly and comprehensively. Even when a smaller volume of comment material is received, the ability to annotate relevant portions and store information about them in a way that permits retrieval and generation of reports can be useful to the agency, especially over time. We describe a prototype application for these purposes. The Workspace for Issue Categorization and Analysis (WICA) allows the rulewriter to create a list of relevant substantive categories and assign them to marked portions of comment text. She can then retrieve all instances of a given issue within the comment pool. Preliminary results of experiments that apply text categorization and active learning methods to comment sets suggest that these techniques can facilitate the marking and category assignment process in lengthy or numerous comment sets. WICA will incorporate these techniques. Other possible applications of WICA within the rulemaking process are discussed.

]]> Thomas R. Bruce et al. A Study in Rule-Specific Issue Categorization for e-Rulemaking http://scholarship.law.cornell.edu/ceri/4 http://scholarship.law.cornell.edu/ceri/4 Mon, 19 Nov 2012 08:39:23 PST We address the e-rulemaking problem of categorizing public comments according to the issues that they address. In contrast to previous text categorization research in e-rulemaking [5, 6], and in an attempt to more closely duplicate the comment analysis process in federal agencies, we employ a set of rule-specific categories, each of which corresponds to a significant issue raised in the comments. We describe the creation of a corpus to support this text categorization task and report interannotator agreement results for a group of six annotators. We outline those features of the task and of the e-rulemaking context that engender both a non-traditional text categorization corpus and a correspondingly difficult machine learning problem. Finally, we investigate the application of standard and hierarchical text categorization techniques to the e-rulemaking data sets and find that automatic categorization methods show promise as a means of reducing the manual labor required to analyze large comment sets: the automatic annotation methods approach the performance of human annotators for both flat and hierarchical issue categorization.

]]> Claire Cardie et al. Facilitative Moderation for Online Participation in eRulemaking http://scholarship.law.cornell.edu/ceri/2 http://scholarship.law.cornell.edu/ceri/2 Fri, 16 Nov 2012 11:38:25 PST This paper describes the use of facilitative moderation strategies in an online rulemaking public participation system. Rulemaking is one of the U.S. government's most important policymaking methods. Although broad transparency and participation rights are part of its legal structure, significant barriers prevent effective engagement by many groups of interested citizens. Regulation Room, an experimental open-government partnership between academic researchers and government agencies, is a socio-technical participation system that uses multiple methods to lower potential barriers to broader participation. To encourage effective individual comments and productive group discussion in Regulation Room, we adapt strategies for facilitative human moderation originating from social science research in deliberative democracy and alternative dispute resolution [24, 1, 18, 14] for use in the demanding online participation setting of eRulemaking. We develop a moderation protocol, deploy it in "live" Department of Transportation (DOT) rulemakings, and provide an initial analysis of its use through a manual coding of all moderator interventions with respect to the protocol. We then investigate the feasibility of automating the moderation protocol: we employ annotated data from the coding project to train machine learning-based classifers to identify places in the online discussion where human moderator intervention is required. Though the trained classifiers only marginally outperform the baseline, the improvement is statistically signifcant in spite of limited data and a very basic feature set, which is a promising result.

]]> Joonsuk Park et al. Knowledge in the People: Rethinking "Value" in Public Rulemaking Participation http://scholarship.law.cornell.edu/ceri/1 http://scholarship.law.cornell.edu/ceri/1 Tue, 13 Nov 2012 11:08:48 PST A companion piece to Rulemaking vs. Democracy: Judging and Nudging Public Participation that Counts, this Essay continues to examine the nature and value of broader public participation in rulemaking. Here, we argue that rulemaking is a “community of practice,” with distinctive forms of argumentation and methods of reasoning that both reflect and embody craft knowledge. Rulemaking newcomers are outside this community of practice: Even when they are reasonably informed about the legal and policy aspects of the agency’s proposal, their participation differs in kind and form from that of sophisticated commenters. From observing the actual behavior of rulemaking newcomers in the Regulation Room project, we suggest that new public participation is often, if not predominantly, experiential in nature and narrative in form. We argue that it is unrealistic to expect that rulemaking newcomers can be significantly inculcated into the norms and methods of the existing rulemaking community of practice. Yet, the potential policymaking value of the on-the-ground, situated knowledge they can bring to the discussion justifies efforts to expand our understanding of the kinds of comments that should “count” in the process. We take some first steps in that direction in this Essay.

]]> Cynthia R. Farina et al.