Different strategies for class model optimization. A comparative study

The Class Modelling (CM) approaches like Soft Independent Modelling of Class Analogy (SIMCA) aim at developing a mathematical model for determination of belongingness of new samples to the studied classes. The main feature of CM is that for each target class an individual model is constructed. CM is widely exploited, e.g., in the food and drug quality testing and authenticity or origin verification. It is well known that the most critical stage in construction of a class model is optimization of its parameters. There exist two basic strategies for optimization of class model, i.e., the "compliant" strategy where the target and nontarget class samples are required in the model optimization process, and the "rigorous" strategy where only the target class samples are used. Since the nontarget class samples are usually available, the compliant scenario is more often explored. In the present study, four different resampling methods for optimization of the SIMCA model (applied in both, a compliant and a rigorous fashion) are thoroughly compared. Each method is tested in combination with two distinct decision threshold estimation criteria: i) an a priori fixing it based on a desired statistical significance level and ii) optimizing it through appropriate data-driven procedures. For the sake of a comprehensive assessment of the studied strategies, several real-world datasets are exploited and final results are post-processed by means of ANalysis Of VAriance (ANOVA). The study reveals that both, a compliant approach with an optimized decision threshold and a rigorous approach with a fixed decision threshold can yield satisfactory classification outcomes, no matter which resampling technique is used. Finally, it is shown how unrepresentativeness of the nontarget classes can lead to the biased classification models when a compliant optimization is carried out. Therefore, a rigorous optimization can be considered as a safer option for the SIMCA model parameter tuning.