Reproducibility

2017 PROSE Award Honorable MentionThe PROSE Awards draw attention to pioneering works of research and for contributions to the conception, production, and design of landmark works in their fields.Featuring peer-reviewed contributions from noted experts in their fields of research, Reproducibility: Principles, Problems, Practices, and Prospects presents state-of-the-art approaches to reproducibility, the gold standard of sound science, from multi- and interdisciplinary perspectives. Including comprehensive coverage for implementing and reflecting the norm of reproducibility in various pertinent fields of research, the book focuses on how the reproducibility of results is applied, how it may be limited, and how such limitations can be understood or even controlled in the natural sciences, computational sciences, life sciences, social sciences, and studies of science and technology.The book presents many chapters devoted to a variety of methods and techniques, as well as their epistemic and ontological underpinnings, which have been developed to safeguard reproducible research and curtail deficits and failures. The book also investigates the political, historical, and social practices that underlie reproducible research in contemporary science studies, including the difficulties of good scientific practice and the ethos of reproducibility in modern innovation societies.Reproducibility: Principles, Problems, Practices, and Prospects is a guide for researchers who are interested in the general and overarching questions behind the concept of reproducibility; for active scientists who are confronted with practical reproducibility problems in their everyday work; and for economic stakeholders and political decision makers who need to better understand the challenges of reproducibility. In addition, the book is a useful in-depth primer for undergraduate and graduate-level courses in scientific methodology and basic issues in the philosophy and sociology of science from a modern perspective. A comprehensive, insightful treatment of the reproducibility challenges facing science today and of ways in which the scientific community can address them. Kathleen Hall Jamieson, Elizabeth Ware Packard Professor of Communication, University of Pennsylvania How can we make sure that reproducible research remains a key imperative of scientific communication under increasing commercialization, media attention, and publication pressure? This handbook offers the first interdisciplinary and fundamental treatment of this important question. Torsten Hothorn, Professor of Biostatistics, University of Zurich Harald Atmanspacher, PhD, is Associate Fellow and staff member at Collegium Helveticum, ETH and University Zurich and is also President of the Society for Mind-Matter Research. He has pioneered advances in complex dynamical systems research and in a number of topics concerned with the relation between the mental and physical. Sabine Maasen, PhD, is Professor for Sociology of Science and Director of the Munich Center for Technology in Society (TU Munich) and Associate Fellow at Collegium Helveticum (ETH and University Zurich). Her research focuses on the interface of science, technology, and society, notably with respect to neuroscience and its applications.

2017 PROSE Award Honorable MentionThe PROSE Awards draw attention to pioneering works of research and for contributions to the conception, production, and design of landmark works in their fields.Featuring peer-reviewed contributions from noted experts in their fields of research, Reproducibility: Principles, Problems, Practices, and Prospects presents state-of-the-art approaches to reproducibility, the gold standard of sound science, from multi- and interdisciplinary perspectives. Including comprehensive coverage for implementing and reflecting the norm of reproducibility in various pertinent fields of research, the book focuses on how the reproducibility of results is applied, how it may be limited, and how such limitations can be understood or even controlled in the natural sciences, computational sciences, life sciences, social sciences, and studies of science and technology.The book presents many chapters devoted to a variety of methods and techniques, as well as their epistemic and ontological underpinnings, which have been developed to safeguard reproducible research and curtail deficits and failures. The book also investigates the political, historical, and social practices that underlie reproducible research in contemporary science studies, including the difficulties of good scientific practice and the ethos of reproducibility in modern innovation societies.Reproducibility: Principles, Problems, Practices, and Prospects is a guide for researchers who are interested in the general and overarching questions behind the concept of reproducibility; for active scientists who are confronted with practical reproducibility problems in their everyday work; and for economic stakeholders and political decision makers who need to better understand the challenges of reproducibility. In addition, the book is a useful in-depth primer for undergraduate and graduate-level courses in scientific methodology and basic issues in the philosophy and sociology of science from a modern perspective."A comprehensive, insightful treatment of the reproducibility challenges facing science today and of ways in which the scientific community can address them." Kathleen Hall Jamieson, Elizabeth Ware Packard Professor of Communication, University of Pennsylvania"How can we make sure that reproducible research remains a key imperative of scientific communication under increasing commercialization, media attention, and publication pressure? This handbook offers the first interdisciplinary and fundamental treatment of this important question."Torsten Hothorn, Professor of Biostatistics, University of ZurichHarald Atmanspacher, PhD, is Associate Fellow and staff member at Collegium Helveticum, ETH and University Zurich and is also President of the Society for Mind-Matter Research. He has pioneered advances in complex dynamical systems research and in a number of topics concerned with the relation between the mental and physical.Sabine Maasen, PhD, is Professor for Sociology of Science and Director of the Munich Center for Technology in Society (TU Munich) and Associate Fellow at Collegium Helveticum (ETH and University Zurich). Her research focuses on the interface of science, technology, and society, notably with respect to neuroscience and its applications.

Contributors ixIntroduction 1Harald Atmanspacher and Sabine MaasenPART I: CONTEXTUAL BACKGROUNDSIntroductory Remarks 9Harald AtmanspacherReproducibility, Objectivity, Invariance 13Holm TetensReproducibility between Production and Prognosis 21Walther ChZimmerliStability and Replication of Experimental Results: A Historical Perspective 39Friedrich SteinleReproducibility of Experiments: Experimenters' Regress, Statistical Uncertainty Principle, and the Replication Imperative 65Harry CollinsPART II: STATISTICAL ISSUESIntroductory Remarks 83Harald AtmanspacherStatistical Issues in Reproducibility 87Werner AStahelModel Selection, Data Distributions and Reproducibility 115Richard Shiffrin and Suyog ChandramouliReproducibility from the Perspective of Meta-Analysis 141Werner EhmWhy Are There so Many Clustering Algorithms, and How Valid Are Their Results? 169Vladimir Estivill-CastroPART III: PHYSICAL SCIENCESIntroductory Remarks 201Harald AtmanspacherFacilitating Reproducibility in Scientific Computing: Principles and Practice 205David H Bailey, Jonathan M Borwein, and Victoria StoddenMethodological Issues in the Study of Complex Systems 233Harald Atmanspacher and Gerda DemmelRare and Extreme Events 251Holger KantzScience under Societal Scrutiny: Reproducibility in Climate Science 269Georg FeulnerPART IV: LIFE SCIENCESIntroductory Remarks 287Harald AtmanspacherFrom Mice to Men: Translation from Bench to Bedside 291Marianne Martic-Kehl and P August SchubigerA Continuum of Reproducible Research in Drug Development 315Gerd Folkers and Sabine BaierRandomness as a Building Block for Reproducibility in Local Cortical Networks 325Johannes Lengler and Angelika StegerNeural Reuse and in-Principle Limitations on Reproducibility in Cognitive Neuroscience 341Michael L AndersonOn the Difference between Persons and Things-Reproducibility in Social Contexts 363Kai VogeleyPART V: SOCIAL SCIENCESIntroductory Remarks 385Sabine Maasen and Harald AtmanspacherOrder Effects in Sequential Judgments and Decisions 391Zheng Wang and Jerome BusemeyerReproducibility in the Social Sciences 407Martin ReinhartAccurate But Not Reproducible? The Possible Worlds of Public Opinion Research 425Felix KellerDepending on Numbers 447Theodore M PorterScience between Trust and Control: Non-Reproducibility in Scholarly Publishing 467Martina FranzenPART VI: WIDER PERSPECTIVESIntroductory Remarks 487Sabine Maasen and Harald AtmanspacherRepetition with a Difference: Reproducibility in Literature Studies 491Ladina Bezzola LambertRepetition Impossible: Co-Affection by Mimesis and Self-Mimesis 511Hinderk EmrichRelevance Criteria for Reproducibility: The Contextual Emergence of Granularity 527Harald AtmanspacherThe Quest for Reproducibility Viewed in the Context of Innovation Societies 541Sabine MaasenIndex 563