Automated Essay Scoring

This book discusses the state of the art of automated essay scoring, its challenges and its potential. One of the earliest applications of artificial intelligence to language data (along with machine translation and speech recognition), automated essay scoring has evolved to become both a revenue-generating industry and a vast field of research, with many subfields and connections to other NLP tasks. In this book, we review the developments in this field against the backdrop of Elias Page's seminal 1966 paper titled "The Imminence of Grading Essays by Computer."

Part 1 establishes what automated essay scoring is about, why it exists, where the technology stands, and what are some of the main issues.In Part 2, the book presents guided exercises to illustrate how one would go about building and evaluating a simple automated scoring system, while Part 3 offers readers a survey of the literature on different types of scoring models, the aspects of essay quality studied in prior research, and the implementation and evaluation of a scoring engine. Part 4 offers a broader view of the field inclusive of some neighboring areas, and Part
ef{part5} closes with summary and discussion.

This book grew out of a week-long course on automated evaluation of language production at the North American Summer School for Logic, Language, and Information (NASSLLI), attended by advanced undergraduates and early-stage graduate students from a variety of disciplines. Teachers of natural language processing, in particular, will find that the book offers a useful foundation for a supplemental module on automated scoring. Professionals and students in linguistics, applied linguistics, educational technology, and other related disciplines will also find the material here useful.

• Preface


• Building an Automated Essay Scoring System


• From Lessons to Guidelines


• Models


• Generic Features


• Genre- and Task-Specific Features


• Automated Scoring Systems: From Prototype to Production


• Evaluating for Real-World Use


• Automated Feedback


• Automated Scoring of Content


• Automated Scoring of Speech


• Fooling the System: Gaming Strategies


• Looking Back, Looking Ahead


• Definitions-in-Context


• Index


• References


• Authors' Biographies