Are the lists of questionable journals reasonable: A case study of early warning journal lists

The use of lists of questionable journals as a means to ensure research quality and integrity is the subject of an ongoing debate due to their ambiguous criteria. To assess the reasonableness of these lists from a typological perspective, we examined how effectively they reflect differences in bibliometric attributes among distinct groups and whether these differences are consistent. Using the Early Warning Journal Lists from the National Science Library of the Chinese Academy of Sciences as a case study, we categorized listed journals by warning levels and publication years. Our findings indicate potential inconsistencies in the criteria used for assigning warning levels, as we observed varying degrees of differences (or their absence) among groups across different key academic indicators. Notably, when it comes to citation metrics like journal impact factor and journal citation indicator, it appears that these criteria may not have been considered for grouping, although this lack of clarity from the creators is apparent. This underscores the importance of conducting more scientific and thorough evaluations of lists of questionable journals, along with a greater emphasis on sharing precise standards and data. Our study also provides recommendations for future iterations of such lists by different institutions.