Over the past decade and a half, a persistent collection of petitioners has pressed the U.S. Fish and Wildlife Service (Service) to list the Bay-Delta population of longfin smelt under the federal Endangered Species Act (ESA). The longfin smelt spawns in freshwater tributaries that feed the Bay, including lower portions of the Sacramento-San Joaquin Rivers Delta. Although the scarcity of longfin smelt in surveys that record pelagic fishes in the Bay and Delta is readily explained — the surveys poorly sample the habitats of longfin smelt during its two-year rearing period and do not at all sample its spawning habitat — data from those surveys have been used to support the notion that the longfin smelt population is greatly imperiled and may be verging toward extinction. Other data and analyses undercut this notion.
Nonetheless, in October 2022, the Service proposed listing the Bay-Delta population of longfin smelt as an endangered species in response to litigation pressuring the agency to act. It relied on population viability analyses (PVAs) that its own staff prepared to support the proposed endangered determination, which were appended to a Species Status Assessment prepared by the agency. The Service claimed in the proposed rule that the “results of the PVAs identified that the probability of quasi-extinction for the Bay-Delta longfin smelt exceeds 20 percent over the next 5 years and reaches 50-60 percent by 2040.” Drawing on those dire predictions, the Service concluded that the population is in danger of extinction throughout its range.
PVAs use data and models to assess the likelihood of persistence of a species over a specified period of time. Unfortunately, the Service made multiple critical errors as it completed and reported the longfin smelt PVAs. Below we identify four obvious and serious errors. Before making a final determination with respect to the longfin smelt listing petition, the Service must issue a new Species Status Assessment and proposed rule that remedy those errors, so that its leadership can make a determination that is informed by “the best scientific and commercial data available” as is required under the ESA.
1 — Technical Note 1 to the Service’s Species Status Assessment details efforts to develop a PVA to predict the probability of extinction of longfin smelt over time. The PVA author uses four quantitative inputs to complete the PVA — the “mean and variance of the population growth rate, the current population size, and a quasi-extinction threshold.” Figure 1 includes the mean and variance of the population growth rate for each survey included in the analysis, as well as a pooled estimate based on all of the individual surveys (a meta-analysis of the mean).
Source: Figure 1 from the PVA in the Species Status Assessment.
There is an unmistakable problem with the data presented in Figure 1 — it is not possible for the meta-analysis mean to be outside the lower and upper bounds of the pooled means for the surveys. This is because that mean is the average of the means for the individual surveys. Any given mean is by definition a mathematical average of two or more numbers; therefore, it is impossible for the mean to be below the lowest number or above the highest number in the series.
2 — A just-published article (Tobias et al. 2023 — Informing extinction risk: Summarizing population viability through a meta-analysis of multiple long-term monitoring programs for a declining estuarine fish species — in the journal Biological Conservation 288:110348) by the author of the longfin smelt PVA highlights the fact that the determination in the proposed rule to list the longfin smelt as endangered is based on a false assertion regarding the probability of quasi-extinction of longfin smelt. As noted above, the proposed rule states that PVA results identify that the probability of quasi-extinction exceeds 20 percent over the next five years. Likewise, the Species Status Assessment states “the probability of quasi-extinction for the Bay-Delta DPS exceeds 20% for all survey time series over the next five years.” In contradistinction, Tobias et al. state plainly and clearly that the probability of quasi-extinction “exceed[s] 20% over two decades.” This four-fold difference from 5 years to 20 years in the estimated time to get to a 20 percent probability of extinction is a material inconsistency between the two treatments of the same data.
3 — Both the PVA described in Technical Note 1 and that in Tobias et al. include a figure that displays estimates of mean population growth rates for longfin smelt based on seven different surveys included in the analysis. The former, Figure 1 from the PVA in the Species Status Assessment, is reproduced above. The latter follows:
Source: Figure 2 from Tobias et al. (2023).
The estimates of the population growth rates differ materially, as do the confidence intervals around those estimates. For example, whereas the estimate of the mean for the 20mm survey in the PVA in the Species Status Assessment indicates a negative population growth rate — that is, a declining population — the estimate of the mean for the same survey in Tobias et al. indicates a positive population growth rate (an increase in longfin smelt numbers). In comparison, the estimate of the mean for the Bay Study Otter Trawl age-0 survey in the PVA indicates a positive population growth rate, while the estimate of the mean for the same survey in Tobias et al. indicates a negative population growth rate. Comparing the figures that purport to report on the same data, but display substantially different means and confidence intervals, raises red flags that the agency has an obligation to address.
4 — Additionally, both the PVA described in Technical Note 1 and that in Tobias et al. include a figure that displays the estimated probability of quasi-extinction.
Source: Figure 1 from the PVA in the Species Status Assessment.
Source: Figure 4 from Tobias et al. (2023).
There is a dramatic difference in the confidence intervals reported. In the former figure from Technical Note 1 to the Species Status Assessment, the confidence interval is so wide that the results cannot inform the determination. What they appear to tell the reader is that authors are 95 percent confident that the probability of extinction in 2040 is between 1 percent and 85 percent. In other words, the analytical results do not provide a precise representation of the mean probability of extinction.
In the latter figure from Tobias et al., the confidence interval reported is narrower, dramatically so. No explanation is provided in Tobias et al. for this difference in confidence intervals. This is particularly perplexing since the figures were produced by the same author using the same data set. Furthermore, the accuracy of the estimate of probability in Tobias et al. is highly questionable, given that the reported confidence intervals remain stable for a period of decades into the future. This is not plausible because, of course, multiple sources of uncertainty invariably increase over time.
Given these errors, the Service should both replicate its prior analyses and show its work in an updated Species Status Assessment and seek peer reviewers to verify independently that both the data are reliable and analyses using the data are appropriate for projecting longfin smelt population sizes. This is all the more important because of the inherent unreliability of the trawl data as a basis for longfin smelt population estimates – an issue raised by multiple peer reviewers of the Species Status Assessment and commentors on the proposed rule. The underlying survey data are plagued by biases and the results that emerge from reliance on those data individually predict population growth ranging from positive to negative. Accordingly, the Service should also re-examine its overall approach to examining likely future viability of the Bay-Delta longfin smelt population and incorporate other data in addition to the longitudinal trawl surveys. By doing so, the agency can assure that its ultimate determination is actually informed by the best available scientific information.
No Comments