Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Aug 4, 2025

Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
07:12

Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss

Published on: April 11, 2025

478

How Trustworthy are Performance Evaluations for Basic Vision Tasks?

Tran Thien Dat Nguyen, Hamid Rezatofighi, Ba-Ngu Vo

    IEEE Transactions on Pattern Analysis and Machine Intelligence
    |April 4, 2023
    PubMed
    Summary
    This summary is machine-generated.

    Related Concept Videos

    You might also read

    Related Articles

    Articles linked to this work by shared authors, journal, and citation graph.

    Sort by
    Same author

    Global survey of secondary metabolism in <i>Aspergillus niger</i> via activation of specific transcription factors.

    PNAS nexus·2025
    Same author

    Acoustic-to-hyper-spectral: real-time perimeter intrusion detection system monitoring through learnable filters and hyper-spectral image generation from distributed acoustic sensing systems.

    Optics express·2025
    Same author

    Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments.

    IEEE transactions on pattern analysis and machine intelligence·2025
    Same author

    Asia-Pacific consensus for the management of osteoporosis in men.

    Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA·2025
    Same author

    Deep learning classification integrating embryo images with associated clinical information from ART cycles.

    Scientific reports·2025
    Same author

    SC-DepthV3: Robust Self-Supervised Monocular Depth Estimation for Dynamic Scenes.

    IEEE transactions on pattern analysis and machine intelligence·2023
    Same journal

    HardFlow: Hard-Constrained Sampling for Flow-Matching Models Via Trajectory Optimization.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Industrial Brain: Self-Evolving Neuro-Symbolic Autonomy with Causal Resilience for Cyber-Physical Systems.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Adaptive Hardness-Driven Dictionary Distillation for Incomplete Streaming View Clustering.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Achieving Text-based Person Retrieval with Any Granularity.

    IEEE transactions on pattern analysis and machine intelligence·2026
    See all related articles

    Evaluating computer vision algorithms for object detection and tracking requires trustworthy performance criteria. This study introduces trustworthiness requirements, including robustness and mathematical consistency, to ensure reliable algorithm evaluations.

    Area of Science:

    • Computer Vision
    • Machine Learning
    • Algorithm Evaluation

    Background:

    • Current performance evaluation criteria for object detection, instance-level segmentation, and multi-object tracking can be unreliable.
    • Algorithm rankings fluctuate with parameter choices (e.g., Intersection over Union threshold), hindering trust in evaluations.

    Purpose of the Study:

    • To introduce a notion of trustworthiness for performance evaluation criteria in computer vision.
    • To establish requirements for trustworthy criteria: robustness, contextual meaningfulness, and mathematical consistency.

    Main Methods:

    • Examined widely-used performance criteria for object detection, instance-level segmentation, and multi-object tracking.
    • Proposed and explored alternative criteria based on metrics for sets of shapes.

    More Related Videos

    Assessing Binocular Central Visual Field and Binocular Eye Movements in a Dichoptic Viewing Condition
    07:45

    Assessing Binocular Central Visual Field and Binocular Eye Movements in a Dichoptic Viewing Condition

    Published on: July 21, 2020

    4.5K
    Methods to Test Visual Attention Online
    09:44

    Methods to Test Visual Attention Online

    Published on: February 19, 2015

    11.9K

    Related Experiment Videos

    Last Updated: Aug 4, 2025

    Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
    07:12

    Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss

    Published on: April 11, 2025

    478
    Assessing Binocular Central Visual Field and Binocular Eye Movements in a Dichoptic Viewing Condition
    07:45

    Assessing Binocular Central Visual Field and Binocular Eye Movements in a Dichoptic Viewing Condition

    Published on: July 21, 2020

    4.5K
    Methods to Test Visual Attention Online
    09:44

    Methods to Test Visual Attention Online

    Published on: February 19, 2015

    11.9K
  • Assessed existing and alternative criteria against the suggested trustworthiness requirements.

    • Main Results:

    • Many widely-used performance criteria overlook essential trustworthiness requirements.
    • Parameter sensitivity and lack of mathematical rigor were observed in common evaluation metrics.
    • Alternative criteria show promise in meeting trustworthiness standards.

    Conclusions:

    • Existing performance evaluation criteria in computer vision often lack trustworthiness.
    • Developing and adopting trustworthy criteria is crucial for reliable algorithm assessment.
    • Future work should focus on robust and mathematically sound evaluation metrics for vision tasks.