GeoToken: Hierarchical Geolocalization of Images via Next Token Prediction

Area of Science:

• Computer Vision
• Machine Learning
• Geographic Information Systems

Background:

• Image geolocalization is challenging due to visual similarities and vast search spaces.
• Existing methods struggle with accuracy and scalability in determining an image's geographic origin.

Purpose of the Study:

• To develop a novel hierarchical sequence prediction approach for accurate image geolocalization.
• To enhance location prediction by mimicking human cognitive strategies for narrowing down geographic areas.

Main Methods:

• Proposed a hierarchical sequence prediction model using S2 cells for nested, multiresolution global grids.
• Employed autoregressive generation, conditioned on visual inputs and previous predictions, similar to language models.
• Investigated inference-time strategies like beam search and multi-sample inference for improved performance.

Main Results:

• Achieved state-of-the-art performance on Im2GPS3k and YFCC4k datasets, surpassing MLLM-free baselines by up to 13.9%.
• Outperformed all baselines when augmented with a Multimodal Large Language Model (MLLM).
• Demonstrated effective uncertainty management through hierarchical path exploration.

Conclusions:

• The proposed hierarchical sequence prediction method significantly advances image geolocalization accuracy.
• The approach offers a robust framework for location determination, adaptable with or without MLLMs.
• This work establishes a new benchmark for image geolocalization tasks.