For the past three decades, the federal and California governments and diverse stakeholders have looked to scientists to explain the causes of the long-term population declines in fish species native to California’s Bay-Delta in an effort to chart a course to conserve those species. Water agencies across the State have invested billions of dollars in data collection and scientific research with the hope of finding clear answers to vexing problems stemming from more than a century of large-scale environmental change to the Pacific Coast’s most expansive estuary. Armed with a sense of urgency, regulatory agencies have felt the compulsion to act. “Clean narratives” sometimes supplied by scientists, but more often created by the regulatory agencies themselves, have provided justifications for such actions. Here, after describing the factors that contribute to a common push for clean narratives, I make the case for scientific honesty and intellectual humility, pointing to recommendations for integration of full disclosure and intellectual humility into the process of reporting on and publishing empirical research.
Scientists – particularly those interested in pursuing a career in academia – have for many decades felt pressure to publish their investigatory work and do so in the most prestigious peer-reviewed journals, giving rise to the adage “publish or perish.” More recently, complementary pressures have provided incentives for scientists to produce what one editorial board referred to as “clean narratives.”
Clean narratives are descriptions of study results that leave little to no room for ambiguity; they purposefully deemphasize the uncertainties that accompany all investigations. Factors that contribute to the push for clean narratives go beyond the pressure to publish or perish and can include the intention to draw attention to one’s work, ignoring the uneasy relationship between science and advocacy, and the difficulty inherent in conveying findings and attending uncertainties to a lay audience.
An example of a clean narrative is that advanced by the authors of a highly publicized study on the status of North American bird populations published in the preeminent journal Science in 2019. The study’s authors, led by Ken Rosenberg at the Cornell Lab of Ornithology, analyzed populations of more than 500 species of birds in North America, estimating a decrease in the combined populations of 2.9 billion birds between 1970 and 2017. Parallel with the release of the study, the authors implemented a sophisticated media strategy that yielded a front-page story in the New York Times on the same day the paper published an op-ed by the study’s authors, and blanket coverage in a wide array of prominent media outlets from National Public Radio to The Atlantic to Scientific American. As one subsequent assessment of the study and its rollout explained, the media strategy included a domain name purchased by the advocacy group American Bird Conservancy, a hashtag, and a YouTube video all intended to present a clear-cut and unequivocal picture of a biodiversity crisis.
While the study’s findings are concerning, any uncertainty or nuance was lost in a calculated media blitz intended to produce headlines, such as “Birds are vanishing from North America,” the lead into the New York Times story on the study. Missing from the media coverage, for example, is the authors’ finding that the species that exhibited the greatest declines are the species that are most abundant. Also, unreported in the major media outlets is the fact many of the species exhibiting large declines are non-native species, including some that humans have made concerted efforts to eliminate. In fact, the one species that exhibited the greatest decline according to the authors, more than 330 million between 1970 and 2017 (or more than 10 percent of the total reported loss), is the non-native house sparrow. And anyone who read the coverage in the New York Times might be surprised to hear that the authors report that 43 percent of species exhibited an increase in abundance, even as 57 percent exhibited a decline in abundance. If the authors had decided to highlight that finding in their media strategy rather than the estimated loss of 3 billion birds, then the media reception likely would have been less enthusiastic, but the coverage would have been more accurate.
Intellectual humility is an essential counterbalance to the temptation to produce clean narratives. This is the case because clean narratives typically overstate the extent and certainty of our knowledge. The history of science teaches us that our understanding of the world around us is always incomplete and, in some cases, flat-out wrong.
Consider the seemingly robust empirical research supporting a broadly accepted view in the medical community that daily aspirin use at low dosages can prevent heart attack or stroke. Recently, reputable media outlets including the New York Times, Washington Post, and National Public Radio reported the findings of a major pooled analysis of six observational studies that led the U.S. Preventive Services Task Force to change its longstanding position regarding the use of aspirin to prevent heart attack or stroke. The analysis and the response of the medical community are noteworthy because literally tens of millions of Americans are taking aspirin daily based on prior analyses of the anticipated benefits as interpreted by the medical community.
This recent, high-profile example of the need for humility is one of innumerable circumstances where the scientific community has advocated for one course of action at one point in time but later changed course. It should serve as a reminder that even when scientists and scientific research appear to explain a certain phenomenon, subsequent research may reveal shortcomings in the contemporary understanding of that phenomenon.
Of course, scientists are taught this from a young age and humility is foundational to the scientific method. But another human trait – arrogance – can blind scientists to the essential role of humility in scientific practice. This is particularly the case as scientists become invested in their work – irrespective of whether the basis for such investment is moral, financial, or otherwise. And at a moment in time when trust in many human institutions is low, but trust in the scientific community remains relatively high, it would be fair to observe that society may be inviting such arrogance.
Societal forces are reinforcing the natural tendency of humans to overestimate their own understanding of the world. The former president perhaps represents the zenith of human arrogance with his audacious claims that he enjoys super-human understanding of issues, such as climate change, Covid-19, matters of national defense, and the deployment of weaponry. But in truth the vast majority (if not all) of us overestimate our understanding of the world around us. Unless scientists are self-consciously vigilant, they may be more apt to fall into this trap than laypersons because of the perception that they actually have acquired the type of expertise that Trump claimed as his in light of their training and research.
The solution to the problem is for the scientific community to embrace precisely the intellectual humility that is at the heart of the scientific method that we learn about in our youth. We must seek to falsify those hypotheses that we accept as explaining phenomena in the world around us. In a recent commentary on intellectual humility in academic research presented in peer-reviewed journals, two psychologists, Rink Hoekstra and Simine Vazire, recommend enhancing the credibility of scientific research by transparently acknowledging limitations and uncertainty. The recommendations, tailored for those engaged in empirical research and seeking its publication in peer-reviewed journals, are to a substantial degree common sense; for example —
· Titles should not imply stronger claims than are justified
· Selective citation should not be used to create a false sense of consistency or conflict in the literature
· Statistical uncertainty that accompanies results should be incorporated into narrative conclusions
· Alternative interpretations should be interpreted in their strongest form
· Press releases and reporting, it should be insisted, must capture the limitations of the work
A good dose of intellectual humility would benefit the Delta’s investigative enterprise and its science program. A telling example is a study published in 2007 with findings that have been misused to establish a series of management actions intended to benefit delta smelt. Fred Feyrer and colleagues analyzed the relationship between three physical environmental factors that vary across the Delta in the autumn – water temperature, Secchi depth (a measure of turbidity), and conductivity (a measure of salinity) – and the distribution of delta smelt and two other fishes. They found that salinity and turbidity explained slightly more than a quarter of the variance in the pattern of delta smelt presences and absences. The authors were forthright about the limitations of their work, stating “[a]lthough we believe our results are robust given the substantial amount of data, we acknowledge that our analysis did not include all potential water quality, physical, or biological factors that affect fish occurrence and habitat.” They went on to state the greatest opportunity for improving upon their analysis lies with additional studies on the effects of prey availability. The inclusion of these acknowledgements of the limitations of the analysis are consistent with the intellectual humility that Hoekstra and Vazire promote.
Despite the acknowledged limitations of the analysis, the Fish and Wildlife Service instituted a regulatory restriction on water supply operations solely on the basis of the work, dictating a specific location for the dynamic low-salinity zone in the Delta in the autumn of above-average and wet years. While the agency acknowledged uncertainties regarding the likely effectiveness of its action and committed to implementing the action in an adaptive management framework, including a robust 10-year review – that review never materialized. In the meantime, in subsequent reports on the action, government scientists have described it as “necessary” to the survival of delta smelt, despite publication of investigations that show the agency’s interpretation of the original study findings were faulty and used an invalid proxy measure for the extent of delta smelt habitat. This demonstrates that even when researchers disclose the limitations of their work, agency scientists and government policymakers may subsequently embrace clean narratives to justify their actions. Intellectual humility is as important in government decision-making at the science and policy interface as it is in scientific research and publication.
The case for intellectual humility has deep roots in history and has for centuries extended to government decision-makers as well as scientists. Cromwell’s Rule, named for the infamous leader of Great Britain in the Seventeenth Century, states that you should never assign a probability of 0 or 1 to anything unless it is logically true or false. This principle, embraced by statisticians, finds its derivation in a speech by Cromwell to the Scots in an effort to avoid war. He pleaded with the General Assembly of the Church of Scotland, “I beseech you, in the bowels of Christ, think it possible that you may be mistaken.” This plea to policymakers, which has informed the work of statisticians and other scientists during the Enlightenment and beyond, is one policymakers and scientists alike should embrace in California’s Bay-Delta and all other contexts where society is grappling with complex conservation challenges. By doing so, our narratives will be less than clean, but we will increase the odds that we can devise effective and timely conservation strategies.
No Comments