InstructPLM-mu: protein fitness prediction from structure instructions

Multimodal protein language models (PLMs) deliver strong performance on mutation-effect prediction, but training such models from scratch demands substantial computational resources. In this paper, we propose a fine-tuning framework called InstructPLM-mu and investigate the question: Can a pretrained sequence-only PLM, augmented with structural information during fine-tuning, match the performance of end-to-end trained multimodal models? Surprisingly, our experiments show that fine-tuning ESM2 with structural inputs can reach performance comparable to ESM3. To understand how this is achieved, we systematically compare three different feature-fusion designs and fine-tuning recipes. Our results reveal that both the fusion method and the tuning strategy strongly affect final accuracy, indicating that the fine-tuning process is not trivial. We hope this work offers practical guidance for injecting structure into pretrained PLMs and motivates further research on better fusion mechanisms and fine-tuning protocols.