Spectral-tokenized prompting of pretrained transformers for industrial NIR hyperspectral impurity segmentation

Near-infrared hyperspectral imaging (NIR-HSI) enables material discrimination through material-dependent spectral signatures, yet industrial impurity segmentation remains challenging due to high spectral dimensionality, limited annotations, and the mismatch with pretrained spatial encoders. Beyond these constraints, severe class imbalance and strict latency requirements in high-throughput production lines further complicate practical deployment. Naively compressing spectra into three channels to fit RGB-pretrained encoders risks discarding essential material cues, whereas training heavy spectral-spatial networks from scratch increases computational cost and overfitting risk under scarce labels. To bridge this mismatch, we propose a Spectral-Tokenized Prompt Transformer (STPT) that injects full-spectrum material cues into a frozen pretrained spatial encoder via lightweight prompting. Specifically, the 288-band hyperspectral cube is tokenized into patch-level spectral prompts using an efficient attention-based prompt encoder, while a pseudo-RGB view is fed into a frozen transformer image encoder to preserve robust spatial priors. The spectral prompts guide spatial embeddings via cross-attention to produce spectrally consistent impurity localization, and a reliability-aware decoding and filtering layer stabilizes predictions under spectral noise and long-tailed distributions. Experiments on a real-world industrial tobacco-stem dataset with 13 labeled categories, including 11 foreign-impurity categories, demonstrate that STPT achieves improved segmentation performance (IoU 0.6949, Dice 0.7377) while meeting the predefined per-patch real-time inference requirement. These results indicate that spectral-tokenized prompting provides an effective and practical mechanism for adapting pretrained vision models to NIR-HSI segmentation under industrial constraints.