An Event-Based Solution to the Perspective-n-Point Problem.

Area of Science:

• Computer Vision
• Robotics
• Neuromorphic Engineering

Background:

• The Perspective-n-Point (PnP) problem is crucial for determining camera pose relative to an object.
• Existing PnP algorithms, typically image-based, often require computationally intensive minimization techniques.
• Neuromorphic event-based vision sensors offer high temporal resolution, presenting an opportunity for more efficient computer vision tasks.

Purpose of the Study:

• To develop the first event-based algorithm for the Perspective-n-Point (PnP) problem.
• To leverage the unique characteristics of neuromorphic sensors for simplified and robust pose estimation.
• To demonstrate the advantages of event-based vision in accelerating computer vision applications.

Main Methods:

• Formulating the PnP problem as a least-squares minimization with event-by-event error updates.
• Computing optimal translation in closed form.
• Determining rotation via the evolution of a virtual mechanical system whose energy equals the error function.
• Proposing two variants: Full and Efficient methods.

Main Results:

• The event-based PnP algorithm achieves pose estimation by incrementally updating with each incoming event.
• The proposed methods demonstrate comparable accuracy to state-of-the-art PnP algorithms.
• Both Full and Efficient methods show significant speed improvements over traditional approaches on synthetic and real-world data.

Conclusions:

• Event-based vision can significantly simplify and accelerate computer vision tasks like PnP.
• The developed algorithm effectively utilizes the high temporal resolution of neuromorphic sensors for real-time pose estimation.
• This work paves the way for more efficient robotic and augmented reality applications leveraging event-based sensing.