Perspectives on Applied Statistics

Assembled by C. Tong. This is a living, growing page, so not the last word.

Richard von Mises

“The leitmotif, the ever recurring melody, is that two things are indispensable in any reasoning, in any description we shape of a segment of reality: to submit to experience and to face the language that is used, with unceasing logical criticism.”

Source: Richard von Mises (1958), Mathematical Theory of Compressible Fluid Flow (New York: Academic Press; reprinted by Dover, 2004), epigraph.

Nassim Nicholas Taleb

“Ludic fallacy (or uncertainty of the nerd): the manifestation of the Platonic fallacy in the study of uncertainty; basing studies of chance on the narrow world of games and dice. A-Platonic randomness has an additional layer of uncertainty concerning the rules of the game in real life.”

Source: Nassim Nicholas Taleb (2010), The Black Swan: The Impact of the Highly Improbable, second edition (New York: Random House), Glossary, p. 303.

John Kay and Mervyn King

“Resolvable uncertainty is uncertainty which can be removed by looking something up (I am uncertain which city is the capital of Pennsylvania) or which can be represented by a known probability distribution of outcomes (the spin of a roulette wheel). With radical uncertainty, however, there is no similar means of resolving the uncertainty – we simply do not know. Radical uncertainty has many dimensions: obscurity; ignorance; vagueness; ambiguity; ill-defined problems; and a lack of information that in some cases but not all we might hope to rectify at a future date. Those aspects of uncertainty are the stuff of everyday experience.

“Radical uncertainty cannot be described in the probabilistic terms applicable to a game of chance. It is not just that we do not know what will happen. We often do not even know the kinds of things that might happen. When we describe radical uncertainty we are not talking about ‘long tails’ – imaginable and well-defined events whose probability can be estimated, such as a long losing streak at roulette. And we are not only talking about the ‘black swans’ identified by Nassim Nicholas Taleb – surprising events which no one could have anticipated until they happen, although these ‘black swans’ are examples of radical uncertainty. We are emphasizing the vast range of possibilities that lie in between the world of unlikely events which can nevertheless be described with the aid of probability distributions, and the world of the unimaginable. This is a world of uncertain futures and unpredictable consequences, about which there is necessary speculation and inevitable disagreement – disagreement which often will never be resolved. And it is that world which we mostly encounter.”

Source: Excerpt from John Kay and Mervyn King (2020), Radical Uncertainty: Decision-Making Beyond the Numbers (New York: W. W. Norton), Ch. 1 (“The Unknowable Future”).

Harry Crane

“Naive probabilism is the (naive) view, held by many technocrats and academics, that all rational thought boils down to probability calculations. It underlies the growing obsession with ‘data-driven methods’ that has overtaken the hard sciences, soft sciences, pseudosciences and non-sciences. It has infiltrated politics, society and business. It’s the workhorse of formal epistemology, decision theory and behavioral economics. Because it is mostly applied in low or no-stakes academic investigations and philosophical meandering, its flaws are easy to overlook. Real world applications of naive probabilism, however, pose disproportionate risks which scale exponentially with the stakes, ranging from harmless (and also helpless) in many academic contexts to destructive in the most extreme cases (war, pandemic). [….]

“The naive probabilist is beholden to the view that rational thinking is probability calculus….Probability calculations are very precise, and for that reason alone they are also of very limited use. Outside of gambling, financial applications, and some physical and engineering problems–and even these are limited–mathematical probability is of little direct use for reasoning with uncertainty. [….]

“Real world problems require more than just thinking in bets. Most common sense is qualitative, thinking about plausibility and possibility long before precisely quantifying probabilities of any kind. Even in cases where the probabilities can be quantified, they should rarely be interpreted literally.”

Source: Exceprts from Harry Crane (2020), “Naive probablism”.

Herbert I. Weisberg

“We must recognize that probability theory alone is insufficient to establish scientific validity. There is only one foolproof way to learn whether an observed finding, however statistically significant it may appear, might actually hold up in practice. We must dust off the time-honored principle of replication as the touchstone of validity. Ideally each study should be validated by collecting new data and performing a new analysis. Only when the system demands and rewards independent replications of study findings can and should public confidence in the integrity of the scientific enterprise be restored.” (p. 344)

“Creating a culture of science in which independent validation becomes a primary criterion for scientific acceptance will help to regain public trust. Knowing that their findings will be subjected to independent scrutiny will impose a higher standard of proof on investigators. At the same time, however, it wil free them up to follow promising leads and refine hypotheses. Requiring real validation will blunt the inhibitions that result from the pressure to achieve statistical significance or bust. Because both data exploration and independent validation require substantial skill and effort, the quantity of research being generated may well decrease as a result. However, the quality should improve greatly.” (p. 363)

Source: Excerpts from Herbert I. Weisberg (2014), Willful Ignorance: The Mismeasure of Uncertainty (Hoboken: Wiley), chapters 11 and 12 (“The Lottery in Science” and “Trust, but Verify”, respectively).

John W. Tukey

“Let us return, then, to the original question: What ought to be the nature of data analysis?

“Data analysis needs to be both exploratory and confirmatory. In exploratory data analysis there can be no substitute for flexibility, for adapting what is calculated – and, we hope, plotted – both to the needs of the situation and the clues that the data have already provided. In this mode, data analysis is detective work – almost an ideal example of seeking what might be relevant.

“Confirmatory data analysis has its place, too. Well used, its importance may even equal that of exploratory data analysis. We dare not, however, let it be an imprimatur or a testimony of infallibility. ‘Not a high priestess but a handmaiden’ must be our demand. Confirmatory data analysis must be the means by which we adjust optimism and pessimism, not only ours but those of our readers. To do this is not easy and may require new approaches and unfamiliar ways of thinking. [….]

“Data analysis has its major uses. They are detective work and guidance counseling. Let us all try to act accordingly.”

Source: Exceprt from the final (“Detective Work versus Sanctification”) section of John W. Tukey (1969), “Data Analysis: Sanctification or Detective Work?”, American Psychologist, 24 (2): 83-91.

David A. Freedman

“Statisticians generally prefer to make causal inferences from randomized controlled experiments, using the techniques developed by [Ronald A.] Fisher and [Jerzy] Neyman. In many situations, of course, experiments are impractical or unethical. Most of what we know about causation in such contexts is derived from observational studies. Sometimes, these are analyzed by regression models; sometimes, these are treated as natural experiments, perhaps after conditioning on covariates. Delicate judgments are required to assess the probable impact of confounders (measured and unmeasured), other sources of bias, and the adequacy of the statistical models used to make adjustments. There is much room for error in this enterprise, and much room for legitimate disagreement.

”[John] Snow’s work on cholera, among other examples, shows that sound causal inferences can be drawn from nonexperimental data. On the one hand, no mechanical rules can be laid down for making such inferences. Since [David] Hume’s day, that is almost a truism. On the other hand, an enormous investment of skill, intelligence and hard work seems to be a requirement. Many convergent lines of evidence must be developed. Natural variation needs to be identified and exploited. Data must be collected. Confounders need to be considered. Alternative explanations have to be exhaustively tested. Above all, the right question needs to be framed.

“Naturally, there is a strong desire to substitute intellectual capital for labor. That is why investigators often try to base causal inference on statistical models. With this approach, P-values play a crucial role. The technology is relatively easy to use and promises to open a wide variety of questions to the research effort. However, the appearance of methodological rigor can be deceptive. Like confidence intervals, P-values generally deal with the problem of sampling error not the problem of bias. Even with sampling error, artifactual results are likely if there is any kind of search over possible specifications for a model, or different definitions of exposure and disease. Models may be used in efforts to adjust for confounding and other sources of bias, but many somewhat arbitrary choices are made. Which variables to enter in the equation? What functional form to use? What assumptions to make about error terms? These choices are seldom dictated either by data or prior scientific knowledge. That is why judgment is so critical, the opportunity for error so large and the number of successful applications so limited.”

Source: Excerpt from the final (“Summary and Conclusions”) section of David Freedman (1999), “From Association to Causation: Some Remarks on the History of Statistics”, Statistical Science, 14 (3): 243-258.

George E. P. Box

“Statistics has no reason for existence except as a catalyst for scientific enquiry in which only the last stage, when all the creative work has already been done, is concerned with a final fixed model and a rigorous test of conclusions. The main part of such an investigation involves an inductive-deductive iteration with input coming from the subject-matter specialist at every stage. This requires a continuously developing model in which the identity of the measured responses, the factors considered, the structure of the mathematical model, the number and nature of its parameters and even the objective of the study change. With its present access to enormous computer power and provocative and thought-provoking graphical display, modern statistics could make enormous contributions to this – the main body of scientific endeavour. But most of the time it does not.”

Source: Excerpt from G. E. P. Box’s Discussion of David Draper (1995), “Assessment and Propagation of Model Uncertainty” (with discussion), Journal of the Royal Statistical Society, Series B, 57 (1): 45–97.

Andrew Gelman

“As a statistician, I make use of the mathematics of probability and variation every day. Outside of my work, though, I am very uncomfortable with uncertainty, whether in the personal or political realms. And I think most people feel the same way. Even in areas such as scientific research or economic and political forecasting where uncertainty is inherent (if you knew what would happen in science, you wouldn’t have to do research; if there was no future uncertainty, there’s be no need to make a forecast), a lot of effort gets put effort into avoiding or denying uncertainty, a ‘premature collapsing of the wave function,’ to use an analogy from quantum physics. When a ‘statistically significant’ result in an experiment is reported as a discovery, or when a non-statistically-significant difference is reported as a null result, this is a denial of uncertainty.

“Collapsing of uncertainty reduces mental tension: it’s work to hold two conflicting ideas in your head at once, and a relief to be able to choose just one—especially if you are persuaded that this choice is justified by science. Hence the appeal of making strong statements, which also can get you some attention and respect if stated with enough of an air of authority….

“The replication crisis in science is a product of systematic discomfort with uncertainty, with speculative results presented as settled fact, leading to distress when these findings do not hold up in later experiments: a scientific and emotional boom-and-bust cycle. Indeed, classical statistical methods seem almost designed to create this boom-and-bust behavior, when non-statistically-significant results are treated as if they were zero and statistically-significant results are taken at face value.”

Source: A. Gelman (2024), The River, the Village, and the Fort: Nate Silver’s new book, On the Edge, Statistical Modeling, Causal Inference, and Social Science, Aug. 13, 2024.

“In statistics and machine learning—theory and application alike—we focus so much on the analysis of some particular dataset in the context of some existing theory. But real science and engineering almost always involves designing new experiments, incorporating new data into our analyses, and trying out new substantive models. From that perspective, looking at post-selection inference is fine—it represents a sort of minimal adjustment of an analysis, in the same way that the sampling standard error from a survey is a minimal statement of uncertainty, representing uncertainty under ideal conditions, not real conditions. [….]

“To me, doing poorly-motivated model selection and then trying to clean it up statistically is kinda like making a big mess and then trying to clean it up, or blowing something up and then trying to put it back together. I’d rather try to do something reasonable in the first place. And then, yes, there are still selection issues—there’s not one single reasonable hierarchical model, or only one single reasonable regularized machine learning algorithm, or whatever—but the selection becomes a much smaller part of the problem, which in practice gets subsumed by multiple starting points, cross validation, new data and theories, external validation, etc.”

Source: A. Gelman (2024:, Answering two questions, one about Bayesian post-selection inference and one about prior and posterior predictive checks, Statistical Modeling, Causal Inference, and Social Science, Dec. 11, 2024.

Erica Thompson

“Though Model Land is easy to enter, it is not so easy to leave. Having constructed a beautiful, internally consistent model and a set of analysis methods that describe the model in detail, it can be emotionally difficult to acknowledge that the initial assumptions on which the whole thing is built are not literally true. This is why so many reports and academic papers about models either make and forget their assumptions, or test them only in a perfunctory way. Placing a chart of model output next to a picture of real observations and stating that they look very similar in all important respects is a statement of faith, not an evaluation of model performance, and any inferences based on that model are still in Model Land.” (Ch. 1)

“In my view, the continued success of modelling depends on creating a programme of understanding that uses models as a tool and as a guide for thinking and communication, and that recognises and is clear about its own limits. As such, one priority is to understand the exits from Model Land and signpost them more clearly….Briefly, there are two exits from Model Land: one quantitative and one qualitative. The quantitative exit is by comparison of the model against out-of-sample data—data that were not used in the construction of the model….The standard disclaimer on investiment opportunities also applies to models: past performance is no guarantee of future success. The qualitative exit from Model Land is much more commonly attempted, but it is also much more difficult to make a successful exit this way. It consists of a simple assertion, based on expert judgment about the quality of the representation, that the model bears a certain relationship with the real world….This is essentially an implicit expert judgment that the model is perfect; Model Land is reality; our assumptions are either literally true or close enough that any differences are negligible….this kind of naive Model Land realism can have catastrophic effects because it invariably results in an underestimation of uncertainties and exposure to greater-than-expected risk….The subjectivity of that second escape route may still worry you. It should.” (Ch. 1)

”[R]eliance on models for information leads to a kind of accountability gap. Who is responsible if a model makes harmful predictions? The notion of ‘following the science’ becomes a screen behind which both decision-makers and scientists can hide, saying ‘the science says we must do X’ in some situations and ‘it’s only a model’ in others. The public are right to be suspicous of the political and social motives behind this kind of dissimulation. Scientists and other authorities must do better at developing and being worthy of trust by making the role of expert judgment much clearer, being transparent about their own backgrounds and interests, and encouraging wider representation of different backgrounds and interests.” (Ch. 1)

“The language of model ‘verification’ and ‘confirmation’…implies that a model can be verified or confirmed to be correct, when in most cases it is a simple category error to treat models as something that can be true or false. […] As statistician George Box famously said, ‘All models are wrong’. In other words, we will always be able to find ways in which models differ from reality, precisely becuae they are not reality. We can invalidate, disconfirm or falsify any model by looking for these differences. Because of this, models cannot act as simple hypotheses about the way in which the true system works, to be accepted or rejected. […] Box’s aphorism has a second part: ‘All models are wrong, but some are useful.’ Even if we take away any philosophical or mathematical justification, we can of course still observe that many models make useful predictions, which can be used to inform actions in the real world with positive outcomes. Rather than claiming, however, that this gives them some truth value, it may be more appropriate to make the lesser claim that a model has been consistent with observation or adequate for a given purpose. Within the range of the available data, we can assess the substance of this claim and estimate the likelihood of further data points also being consistent. Models that are essentially interpolatory, where the observations do not stray much outside the range of the data used to generate the models, can do extremely well by these methods. The extrapolatory question, of the extent to which it will continue to be consistent with observation outside the range of the available data, is entirely reliant on the subjective judgment of the modeller.” (Ch. 2)

Source: Excerpts from Erica Thompson’s Escape from Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It (2022, New York: Basic Books).

General Colin L. Powell

“When I left the A Shau Valley, I shifted from a worm’s-eye to a bird’s-eye view of the war, and the new vantage point was not comforting. One of my assignments was to feed data to a division intelligence officer who was trying to predict when mortar attacks were most likely to occur. He worked behind a green door marked ‘No Entry’ doing something called ‘regression analysis’. My data got through the door, but not me. I was not cleared to enter. One day, the officer finally emerged. There were, he reported, periods when we could predict increased levels of mortar fire with considerable certainty. When was that? By the dark of the moon. Well, knock me over with a rice ball. Weeks of statistical anaylsis had taught this guy what any ARVN private could have told him in five seconds. It is more dangerous out there when it is dark….Deep thinkers, like my intelligence officer behind the green door, were producing printouts, filling spreadsheets, crunching numbers, and coming out with blinding flashes of the obvious, while an enemy in black pajamas and Firestone flip-flops could put an officer out of the war with a piece of bamboo dipped in manure….”

“I began developing another rule: don’t be buffaloed by experts and elites. Experts often possess more data than judgment. Elites can become so inbred that they produce hemophiliacs who bleed to death as soon as they are nicked by the real world.”

Source: Excerpts from Chapter 4, My American Journey, by Colin Powell with Joseph E. Persico (1995, New York: Random House).

Elisabeth Labrousse, on Pierre Bayle

“It is not easy to formulate any kind of conclusion about someone who so delighted in leaving questions open, adopted so deliberately flippant a tone, displayed his pessimism so cheerfully (life being much too tragic to be taken seriously) and wore his massive erudition as lightly as Bayle did….Truth, he held, is not a body of knowledge that can be handed down, by ancestors, priests or rulers. It is something one has to discover for oneself and make one’s own, and this necessarily makes it subjective, finite and liable to the influence of ignorance and error. It has to be thought of as the object of a permanent quest, a goal that no human being can ever actually reach.”

Source: Excerpt from the Conclusion (Chapter 6) of E. Labrousse’s Bayle (1983), translated by Denys Potts (New York: Oxford University Press).

Richard P. Feynman

“[T]he imagination of nature is far, far greater than the imagination of man.”

Source: The Value of Science, lecture at the Autumn 1955 meeting of the National Academy of Sciences. Printed in CalTech’s Engineering and Science magazine, Dec. 1955 issue, vol. 19, no. 3, pp. 13-15.

“It is necessary and true that all of the things we say in science, all of the conclusions, are uncertain, because they are only conclusions. They are guesses as to what is going to happen, and you cannot know what will happen, because you have not made the most complete experiments….

“Scientists, therefore, are used to dealing with doubt and uncertainty. All scientific knowledge is uncertain. This experience with doubt and uncertainty is important. I believe that it is of very great value, and one that extends beyond the sciences. I believe that to solve any problem that has never been solved before, you have to leave the door to the unknown ajar. You have to permit the possibility that you do not have it exactly right. Otherwise, if you have made up your mind already, you might not solve it.

“When the scientist tells you he does not know the answer, he is an ignorant man. When he tells you he has a hunch about how it is going to work, he is uncertain about it. When he is pretty sure of how it is going to work, and he tells you, ‘This is the way it’s going to work, I’ll bet,’ he still is in some doubt. And it is of paramount importance in order to make progress, that we recognize this ignorance and this doubt. Because we have the doubt, we then propose looking in new directions for new ideas. The rate of the development of science is not the rate at which you make observations alone but, much more important, the rate at which you create new things to test.

“If we were not able or did not desire to look in any new direction, if we did not have a doubt or recongize ignorance, we would not get any new ideas. There would be nothing worth checking, because we would know what is true. So what we call scientific knowledge today is a body of statements of varying degrees of uncertainty. Some of them are most unsure; some of them are nearly sure; but none is absolutely certain. Scientists are used to this. We know that it is consistent to be able to live and not know….

“This freedom to doubt is an important matter in the sciences and, I believe, in other fields. It was born of a struggle. It was a struggle to be permitted to doubt, to be unsure. And I do not want us to forget the importance of the struggle and, by default, to let the thing fall away. I feel a responsibility as a scientist who knows the great value of a satisfactory philosophy of ignornace, and the progress made possible by such a philosophy, progress which is the fruit of freedom of thought. I feel a responsibility to proclaim the value of this freedom and to teach that doubt is not to be feared, but that it is to be welcomed as the possibility of a new potential for human beings. If you know that you are not sure, you have a chance to improve the situation. I want to demand this freedom for future generations.”

Source: Excerpt from Richard Feynman’s John Danz lectures at the University of Washington, April 1963; printed in The Meaning of it All: Thoughts of a Citizen-Scientist (Perseus, 1998), end of chapter 1, “The Uncertainty of Science”.

Ibn al-Haytham

“Truth is sought for itself; and in seeking that which is sought for itself one is only concerned to find it. To find the truth is hard and the way to it rough. For the truths are immersed in uncertainties, and all men are naturally inclined to have faith in the scientists. Thus when a man looks into the writings of scientists and, following his natural inclination, confines himself to grasping their pronouncements and intentions, the truth [for him] will consist of their intended notions and their indicated goals. But God has not protected scientists from error, nor has He made their science immune from shortcomings and defects. Had this not been the case, they would not have disagreed about anything in the sciences, nor would their opinions have differed in regard to the true nature of things. But the facts are otherwise. The seeker after the truth is, therefore, not he who studies the writings of the ancients and, following his natural disposition, puts his trust in them, but rather the one who suspects his faith in them and questions what he gathers from them, the one who submits to argument and demonstration, and not to the sayings of a human being whose nature is fraught with all kinds of imperfection and deficiency. It is thus the duty of the man who studies the writings of scientists, if learning the truth is his goal, to make himself an enemy of all that he reads, and, applying his mind to the core and margins of its content, attack it from every side. He should also suspect himself as he performs his critical examination of it, so that he may avoid falling into either prejudice or leniency. If he follows this path the truths will be revealed to him, and whatever shortcomings or uncertainties may exist in the discourse of those who came before him will become manifest.”

Sourrce: Abu Ali al-Hasan ibn al-Hasan ibn al-Haytham (a.k.a., AlHazen), Doubts on Ptolemy (11th century). Translated by A. I. Sabra in his Commentary on Book I of The Optics of Ibn al-Haytham, Books I-III (London: Warburg Institute, 1989),

Required reading for all applied statisticians and data professionals (IMHO)

D. G. Altman and J. M. Bland, 1995: Absence of evidence is not evidence of absence. British Medical Journal, 311: 485.

V. Amrhein, D. Trafimow, and S. Greenland, 2019: Inferential statistics as descriptive statistics: there is no replication crisis if we don’t expect replication. The American Statistician, 73 (sup1): 262-270.

G. E. P. Box, 1976: Science and statistics. Journal of the American Statistical Association, 71: 791-799.

G. E. P. Box, 2001: Statistics for discovery. Journal of Applied Statistics, 28: 285-299.

A. Bradford Hill, 1965: The environment and disease: association or causation? Proceedings of the Royal Society of Medicine, 58 (5): 295-300.

L. Breiman, 2001: Statistical modeling: The two cultures (with discussion). Statistical Science, 16: 199-231.

C. Chatfield, 1995: Model uncertainty, data mining and statistical inference (with discussion). Journal of the Royal Statistical Society, Series A, 158: 419-466.

W. E. Deming, 1975: On probability as a basis for action. The American Statistician, 29: 146-152.

P. Diaconis, 1985: Theories of data analysis: From magical thinking through classical statistics. In Exploring Data Tables, Trends, and Shapes, ed. by D. C. Hoaglin, F. Mosteller, and J. W. Tukey (New York: Wiley), 1-36.

D. Donoho, 2017: 50 years of data science (with discussion). Journal of Computational and Graphical Statistics, 26: 745-785.

A. S. C. Ehrenberg, 1990: A hope for the future of statistics: MSOD. The American Statistician, 44: 195-196.

A. S. C. Ehrenberg and J. A. Bound, 1993: Predictability and prediction. Journal of the Royal Statistical Society, Series A, 156: 167-206.

W. Feller, 1970: An Introduction to Probability Theory and Its Applications, revised third edition (New York: Wiley), Introduction (pp. 1-6).

D. A. Freedman, 1999: From association to causation: some remarks on the history of statistics. Statistical Science, 14: 243-258.

J. H. Friedman, 2001: The role of statistics in the data revolution? International Statistical Review, 69: 5-10.

A. Gelman and E. Loken, 2014: The statistical crisis in science. American Scientist, 102: 460-465.

G. Gigerenzer and J. Marewski, 2015: Surrogate science: The idol of a universal method for scientific inference. Journal of Management, 41: 421-440.

S. Greenland, 2017: For and against methodologies: some perspectives on recent causal and statistical inference debates. European Journal of Epidemiology, 32: 3-20.

S. Greenland, 2017: The need for cognitive science in methodology. American Journal of Epidemiology, 186: 639-645.

B. Hayes, 2001: Randomness as a resource. American Scientist, 89: 300-304.

J. R. Hollenbeck and P. M. Wright, 2016: Harking, sharking, and tharking: Making the case for post hoc analysis of scientific data. Journal of Management, 43: 5-18.

C. Krumme, 2017: Babylonian lottery. Edge.

M. Lavine, 2019: Frequentist, Bayes, or other? The American Statistican, 73 (sup1): 312-318.

G. J. Lithgow, M. Driscoll, and P. Phillips, 2017: A long journey to reproducible results. Nature, 548: 387-388.

J. J. Locascio, 2019: The impact of results blind scientific publishing on statistical consulation and collaboration. The American Statistician, 73 (sup1): 346-351.

B. B. McShane, D. Gal, A. Gelman, C. Robert, and J. L. Tackett, 2019: Abandon statistical significance. The American Statistician, 73 (sup1): 235-245.

C. Mallows, 1998: The zeroth problem. The American Statistician, 52: 1-9.

J. S. Mogil and M. R. Macleod, 2017: No publication without confirmation. Nature, 542: 409-411.

M. R. Munafo and G. Davey Smith, 2018: Robust research needs many lines of evidence. Nature, 553: 399-401.

H. Quinn, 2009: What is science? Physics Today, 62 (7): 8-9.

P. S. Reynolds, 2022: Between two stools: preclinical research, reproducibility, and statistical design of experiments. BMC Research Notes, 15: 73.

M. Schrage, 2000: How the bell curve cheats you. Fortune, 21 February 2000, p. 296.

C. Seife, 2000: CERN’s gamble shows perils, rewards of playing the odds. Science, 289: 2260-2262.

L. Shepp, 2007: Statistical thinking: From Tukey to Vardi and beyond. In Complex Datasets and Inverse Problems: Tomography, Networks and Beyond, ed. by R. Liu, W. Strawderman, and C.-H. Zhang. IMS Lecture Notes-Monograph Series, 54: 268-273.

J. P. Simmons, L. D. Nelson, and U. Simonsohn, 2011: False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22: 1359-1366.

E. L. Thompson and L. A. Smith, 2019: Escape from model-land. Economics, 13: 2019-40.

J. W. Tukey, 1969: Analyzing data: sanctification or detective work? American Psychologist, 24: 83-91.

J. W. Tukey, 1980: We need both exploratory and confirmatory. American Statistician, 34: 23-25.

R. L. Wasserstein and N. A. Lazar, 2016: The ASA’s statement on p-values: context, process, and purpose. The American Statistician, 70: 129-133.

Some thoughts on being a researcher

Allen V. Astin’s invited address to the American Physical Society (APS), Washington, D.C., May 1, 1953.

Richard W. Hamming’s lecture, You and Your Research. There are several versions floating around the Internet, including video recordings such as this, but a somewhat definitive version can be found in Hamming’s book The Art of Doing Science and Engineering: Learning to Learn (Gordon and Breach, Amsterdam, 1997), chapter 30.

Eugene N. Parker’s article, “The martial art of scientific publication”, EOS, Transactions of the American Geophysical Union, vol. 78, no. 37, pp. 391-395.

Freeman Dyson’s never delivered AMS Einstein Lecture, Birds and Frogs, published in 2009 in the Notices of the American Mathematical Society, vol. 56, no. 2, pp. 212-223.

The Scientific Virtues by Slime Mold Time Mold (Feb. 10, 2022).