Guidance and Considerations When Performing Data-Validity Checks

This response to a Commentary by Delgado-Ron, Jeyabalan, Watt, and Salway (2024) on Cimpian, Timmer, and Kim's (2023) paper discusses and clarifies some key issues in applying the new data-validity sensitivity analysis proposed by Cimpian et al. (2023). The differences in the applications of the method by Delgado-Ron et al. (2024) and Cimpian et al. (2023) present an opportunity to recognize the possibilities of this method, while also noting some challenges and limitations. This response-commentary focuses on five key areas: (1) surmising a set of likely motivations of the survey respondents, (2) selecting the screener items, (3) considering the outcomes examined, (4) reflecting on null results in a sensitivity analysis, and (5) recognizing the advantages and disadvantages of the ease of this new method.