Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Aug 4, 2025

Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
07:12

Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss

Published on: April 11, 2025

478

How Trustworthy are Performance Evaluations for Basic Vision Tasks?

Tran Thien Dat Nguyen, Hamid Rezatofighi, Ba-Ngu Vo

    IEEE Transactions on Pattern Analysis and Machine Intelligence
    |April 4, 2023
    PubMed
    Summary
    This summary is machine-generated.

    Related Concept Videos

    You might also read

    Related Articles

    Articles linked to this work by shared authors, journal, and citation graph.

    Sort by
    Same author

    Global survey of secondary metabolism in <i>Aspergillus niger</i> via activation of specific transcription factors.

    PNAS nexus·2025
    Same author

    Acoustic-to-hyper-spectral: real-time perimeter intrusion detection system monitoring through learnable filters and hyper-spectral image generation from distributed acoustic sensing systems.

    Optics express·2025
    Same author

    Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments.

    IEEE transactions on pattern analysis and machine intelligence·2025
    Same author

    Asia-Pacific consensus for the management of osteoporosis in men.

    Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA·2025
    Same author

    Deep learning classification integrating embryo images with associated clinical information from ART cycles.

    Scientific reports·2025
    Same author

    SC-DepthV3: Robust Self-Supervised Monocular Depth Estimation for Dynamic Scenes.

    IEEE transactions on pattern analysis and machine intelligence·2023
    Same journal

    Relation DETR+: Exploring Explicit Position Relation Prior for Dense Prediction.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoning.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    CAFE: Cross-View Adaptive Fusion and Cluster Center Enhancement for Robust Multi-View Clustering.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Learning Shape Anchors for Holistic Indoor Scene Understanding.

    IEEE transactions on pattern analysis and machine intelligence·2026
    See all related articles

    Evaluating computer vision algorithms for object detection and tracking requires trustworthy performance criteria. This study introduces trustworthiness requirements, including robustness and mathematical consistency, to ensure reliable algorithm evaluations.

    Area of Science:

    • Computer Vision
    • Machine Learning
    • Algorithm Evaluation

    Background:

    • Current performance evaluation criteria for object detection, instance-level segmentation, and multi-object tracking can be unreliable.
    • Algorithm rankings fluctuate with parameter choices (e.g., Intersection over Union threshold), hindering trust in evaluations.

    Purpose of the Study:

    • To introduce a notion of trustworthiness for performance evaluation criteria in computer vision.
    • To establish requirements for trustworthy criteria: robustness, contextual meaningfulness, and mathematical consistency.

    Main Methods:

    • Examined widely-used performance criteria for object detection, instance-level segmentation, and multi-object tracking.
    • Proposed and explored alternative criteria based on metrics for sets of shapes.

    More Related Videos

    Assessing Binocular Central Visual Field and Binocular Eye Movements in a Dichoptic Viewing Condition
    07:45

    Assessing Binocular Central Visual Field and Binocular Eye Movements in a Dichoptic Viewing Condition

    Published on: July 21, 2020

    4.5K
    Methods to Test Visual Attention Online
    09:44

    Methods to Test Visual Attention Online

    Published on: February 19, 2015

    11.9K

    Related Experiment Videos

    Last Updated: Aug 4, 2025

    Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
    07:12

    Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss

    Published on: April 11, 2025

    478
    Assessing Binocular Central Visual Field and Binocular Eye Movements in a Dichoptic Viewing Condition
    07:45

    Assessing Binocular Central Visual Field and Binocular Eye Movements in a Dichoptic Viewing Condition

    Published on: July 21, 2020

    4.5K
    Methods to Test Visual Attention Online
    09:44

    Methods to Test Visual Attention Online

    Published on: February 19, 2015

    11.9K
  • Assessed existing and alternative criteria against the suggested trustworthiness requirements.

    • Main Results:

    • Many widely-used performance criteria overlook essential trustworthiness requirements.
    • Parameter sensitivity and lack of mathematical rigor were observed in common evaluation metrics.
    • Alternative criteria show promise in meeting trustworthiness standards.

    Conclusions:

    • Existing performance evaluation criteria in computer vision often lack trustworthiness.
    • Developing and adopting trustworthy criteria is crucial for reliable algorithm assessment.
    • Future work should focus on robust and mathematically sound evaluation metrics for vision tasks.