iPheno: A novel dual-aware vision-language model for multi-task fine-scale crop phenotyping

Crop phenotyping is crucial for advancing plant breeding yet remains a significant challenge. Manual approaches are labor-intensive and do not scale to the analysis of large datasets, while computational methods such as vision-language models (VLMs) lack the adaptability needed for fine-scale spatial reasoning and diverse phenotyping scenarios. To address these gaps, we present iPheno, a fully open-source, domain-specialized multimodal VLM for fine-scale crop phenotyping. A key innovation of iPheno is its dual-aware architecture. First, a spatial-aware feature extractor samples masked regions to construct local K-nearest neighbor graphs and enables fine-scale analysis of arbitrary-shaped regions. Second, a task-aware mixture-of-experts (MoE) routing mechanism activates specialized modules for each phenotyping task. To train iPheno and enable rigorous benchmarking, we constructed iPheno-120K, a large-scale, high-precision dataset designed for multiple phenotyping tasks. Evaluations on the iPheno-120K test set and other publicly available datasets showed that iPheno outperformed all fine-tuned baselines, improving the F1-score by 17.1% (relative to LLaVA-1.6-13B) to 28.9% (relative to MiniCPM-o-9B), while achieving the highest inference speed and memory efficiency. A web server (https://ipheno.ai4bread.com), a mobile application (www.ipheno.cn), and a stand-alone PC client (https://github.com/2997029323/iPheno-PC-Client) are available for iPheno.