Entropy-based byte patching transformer for self-supervised pretraining of SMILES data

Efficient molecular representations are critical for improving the performance and generalization of large language models in chemical learning. Transformer-based architectures have advanced molecular representation learning, yet capturing localized and hierarchical chemical structures remains challenging. We introduce the SMILES Byte-Patch Transformer (SMiBPT), an adaptive model that dynamically segments SMILES and DeepSMILES strings into chemically meaningful substructures through entropy-based byte patching. Unlike fixed and traditional tokenization, our method adjusts patch sizes via entropy thresholds to preserve molecular semantics and enhance feature extraction. SMiBPT integrates self-supervised pretraining, chemical motif-aware encoding, adaptive entropy-aware masking, and rotary position embeddings to model ∼216 million unlabeled molecules from PubChem without truncation. We analyze entropy patterns across molecular classes and optimize patching by chemical properties such as aromaticity, charge, and functional groups. Through large-scale pretraining on ∼1.2 billion patches and extensive ablation studies, SMiBPT, despite its parameter-efficient transformer architecture, outperforms ChemBERTa, SMILES-BERT, and MoLFormer in predictive accuracy, efficiency, and zero-shot transfer analysis.