Boosting pre-trained model with silica nanoparticles cellular toxicity prediction

Silica nanoparticles have been widely adopted as carriers for drug delivery and components of multifunctional nanocomposites, but potentially lead to off-target accumulation and subsequent cytotoxic effects. Previous works explored data-driven methods to improve the evaluation efficiency and supporting the rational design of nanomedicines. However, two challenges still need to be considered. The first is data leakage problem, as previous methods incorporate either evaluation stage features (e.g., Viability_indicator, Positive_control) or rely on one-hot encoding that requires prior knowledge of all categorical values, leading to data leakage risk. Second, the model has poor generalization ability. One-hot encoding fixes the dimensionality of the feature space, causing the model to fail when faced with unseen class values. In this work, we propose a pre-trained model based framework for silica nanoparticles Cellular Toxicity Prediction. To address the data leakage problem, we first removed features that come from the drug evaluation stage such as Viability_indicator, Positive_control, SiO₂NP_label, Interference_testing, and Assay_viability. And then we utilize the embedding layer from the TabPFN to process the original categorical values into dense vectors. To improve the model generalizability, we employ in-context learning on pre-trained TabPFN, which has already learned a large number of patterns from amount of synthetic data. The TabPFN model only needs to adapt the output prediction distribution through in-context learning. Experimental results on publicly available dataset demonstrate that our framework not only achieves state-of-the-art classification performance but also effectively mitigates data leakage and improves generalizability for novel nanoparticle formulations. The code and data are shared in https://github.com/AppleMax1992/pre-trained_nanosilica .