“Non-reproducible single occurrences are of no significance to science.”
Karl Popper (1959) “The logic of scientific discovery”, p. 66
“We may say that a phenomenon is experimentally demonstrable when we know how to conduct an experiment which will rarely fail to give us statistically significant results.”
Ronald Fisher (1935) “The Design of Experiments”, p. 14
The view that “it’s science only if we can repeatedly demonstrate it” is deeply ingrained into modern scientific thought, as exemplified by the above citations from Popper and Fisher. However, rarely this dogma is well-formulated, and thus it is not always clear what are the criteria for true replication.
Experimental scientific papers are required to include a clear Methods section, so as to enable other scientists to repeat the experiment and replicate the result. Who established this dogma and when? It was the British physicist Robert Boyle in the 17th century, during his debate with philosopher Thomas Hobbs over his air pump and the nature of vacuum. This debate is described in a book by Shapin and Schaffer, and more briefly by Colin Baxter.
As Baxter describes, Boyle maintained that the foundations of knowledge should be constituted by experimentally produced facts. By repeating the same experiment over and over again, the certainty of fact will emerge. When should we stop experimenting and know that we have a fact? Boyle proposed that this was to be established by the aggregation of individuals’ beliefs, members of an intellectual collective (for Boyle, this meant the Royal Society). In this way, Boyle maintained that a scientific community could be built up on the basis of reporting experiments, and all inside the community could accept them as factual. The modern style of scientific papers – dry and precise, written in the vernacular (i.e., not in Latin) – was developed and mastered by Boyle as a mean to convince such a community that the author is a sober, methodical person who can be believed, even when he or she makes claims that are generally considered controversial.
Boyle’s air pump was the scientific apparatus that produced vacuum – at that time a controversial concept that many distinguished philosophers considered impossible. The air pump was also complicated and expensive to build, the “big science” of its period. Shapin and Schaffer describe how “Christiaan Huygens was the only natural philosopher in the 1660s who built an air-pump that was outside the direct management of Boyle and Hooke”. When Huygens noticed an effect that was not noticed by Boyle “it became clear that unless the phenomenon could be produced in England with one of the two pumps available, then no one in England would accept the claims Huygens had made, or his competence in working the pump”. Replicability became the gold standard of science. Yet Shapin and Schaffer also note that “the accomplishment of replication was dependent on contingent acts of judgment. One cannot write down a formula saying when replication was or was not achieved”.
References
Popper, K. R. (1959). The logic of scientific discovery. Hutchinson, London, United Kingdom.
Steven Shapin and Simon Schaffer (1985). Leviathan and the Air-Pump. Princeton University Press, Princeton, New Jersey.
Colin Baxter (2008) Leviathan Against the Air Pump.
