SpatialPrompting: pose-aware keyframe prompting for 3D spatial QA toward smart indoor environments

SpatialPrompting is a practical, training-free and model-agnostic framework for 3D spatial reasoning with off-the-shelf multimodal large language models, requiring no fine-tuning or 3D-specific inputs. It adopts a pose-aware, keyframe-driven prompting strategy: we select a compact, diverse set of frames using vision-language similarity, Mahalanobis distance, field of view, and image sharpness, and then verbalize each camera pose to enable multi-view, viewpoint-aware reasoning within a single structured prompt. Evaluations on ScanQA and SQA3D using GPT-4o and Gemini show that SpatialPrompting achieves competitive performance on ScanQA compared to existing approaches, while remaining slightly below the best fine-tuned methods on SQA3D under a training-free setting. On our Complex Spatial QA (CSQA) dataset, the proposed method improves accuracy from 62 to 78 (+16) over a GPT-4o baseline and consistently outperforms query-based keyframe selection. Furthermore, experiments across multiple models-including GPT-4o, Gemini, and Qwen-demonstrate that the framework generalizes across both proprietary and open-source models. By eliminating task-specific training and enabling reuse across different models without retraining, SpatialPrompting shifts the cost from model-specific training to flexible inference, offering a scalable and effective approach for real-world spatial reasoning as multimodal models continue to evolve.