Can I Trust This Location Estimate? Reproducibly Benchmarking the Methods of Dynamic Accuracy Estimation of...

Area of Science:

• Computer Science
• Geomatics Engineering
• Electrical Engineering

Background:

• Limited research exists on the confidence metrics used by indoor positioning systems (IPS).
• Dynamic accuracy estimation (DAE) for IPS is under-explored, with recent attention in outdoor IoT LPWAN systems.
• Inconsistent comparisons, lack of baselines, varied metrics, and absence of open data/code hinder progress in DAE for positioning.

Purpose of the Study:

• To present an open-source, reproducible benchmarking framework for evaluating and comparing DAE methods in IPS.
• To establish a consistent terminology, discuss baselines, and define evaluation metrics for DAE.
• To evaluate multiple DAE methods using open data, code, and a comprehensive set of metrics.

Main Methods:

• Developed an open-source, reproducible benchmarking framework.
• Conducted a literature review to define terminology, baselines, and metrics for DAE.
• Evaluated various DAE methods using open datasets and code.

Main Results:

• The framework enables consistent and transparent evaluation of DAE methods.
• Multiple DAE methods were assessed using a rich set of metrics.
• Established a baseline for comparing DAE techniques in IPS and LPWAN positioning.

Conclusions:

• The presented framework addresses the need for reproducible and consistent evaluation of DAE methods.
• This work sets the state-of-the-art for DAE in IPS and LPWAN positioning systems.
• Promotes open science practices, facilitating future research and development in positioning accuracy estimation.