The ghost of selective inference in spatiotemporal trend analysis

In spatiotemporal trend analysis, selective inference occurs when researchers are only interested in significant trends based on a fixed threshold (α, often 0.05), without considering the total number of statistical tests performed. Using simultaneous inference in gridded data involves thousands of trend tests, one for each pixel, leading to multiple testing or multiplicity problems. Multiplicity increases the chance of false discoveries in an unknown way unless the p-values of all tests performed are appropriately considered and adjusted. This discussion paper provides a selective and non-exhaustive review of the problems of multiplicity and selective inference. We discuss some appropriate methods to cope with the inflation of spurious results and comment on some examples based on gridded data in the context of research on spatiotemporal trend analysis. In addition, we suggest some good practices in transparency to facilitate the replicability of studies. The effects of uncorrected multiplicity and selective interference can be likened to a ghostly layer over the data, projecting illusions of significance that vanish with rigorous correction methods, revealing the true statistical skeleton of the results. The basis for addressing these problems is to assume that, although it may sometimes seem counterintuitive, the reality of what we perceive as statistically significant (i.e., p-values <0.05) also depends on the number (and value) of what we perceive as non-significant (i.e., p-values ≥0.05). Indeed, in a multiplicity context, one cannot correctly decide what is statistically significant until the whole story is known. Uncorrected selective inference precisely involves ignoring part of the story.