Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Heuristics01:21

Heuristics

46
Heuristics are problem-solving strategies that use mental shortcuts to simplify decision-making. Unlike algorithms, which must be followed precisely to achieve a correct result, heuristics offer a general problem-solving framework. They save time and energy but can sometimes lead to less rational decisions.
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...
46
Decision Making01:20

Decision Making

69
Decision-making is a fundamental cognitive process that involves evaluating alternatives and selecting among them. This process can range from simple choices, such as deciding what to wear, to complex decisions, like choosing a major in college or a career path. The complexity of the decision often dictates the approach we use, which can be broadly categorized into two types: automatic and controlled decision-making.
Automatic decision-making is fast, intuitive, and relies on gut feelings...
69
The Availability Heuristic01:08

The Availability Heuristic

5.8K
A heuristic is a general problem-solving framework (Tversky & Kahneman, 1974). You can think of these as mental shortcuts that are used to solve problems. Different types of heuristics are used in different types of situations, and the impulse to use a heuristic occurs when one of five conditions is met (Pratkanis, 1989):
5.8K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Venous Cannulation Site Stenosis and Risk of Recurrence: A New Hypothesis.

Cureus·2025
Same author

A Measure of Synergy Based on Union Information.

Entropy (Basel, Switzerland)·2024
Same author

Orders between Channels and Implications for Partial Information Decomposition.

Entropy (Basel, Switzerland)·2023
Same author

A classification-based approach to semi-supervised clustering with pairwise constraints.

Neural networks : the official journal of the International Neural Network Society·2020
Same author

External Patch-Based Image Restoration Using Importance Sampling.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2019
Same author

A Convergent Image Fusion Algorithm Using Scene-Adapted Gaussian-Mixture-Based Denoising.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2018

Related Experiment Video

Updated: May 10, 2025

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents
07:42

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents

Published on: August 2, 2018

13.4K

A benchmarking framework and dataset for learning to defer in human-AI decision-making.

Jean V Alves1, Diogo Leitão2, Sérgio Jesus2

  • 1Feedzai, Coimbra, Portugal. jean.alves@feedzai.com.

Scientific Data
|April 23, 2025
PubMed
Summary

Learning to Defer (L2D) algorithms enhance human-AI collaboration. A new framework, OpenL2D, generates realistic synthetic experts for better L2D system testing, revealing performance variations based on expert diversity.

More Related Videos

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents
07:05

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Published on: September 10, 2018

5.9K
Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
06:37

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

2.4K

Related Experiment Videos

Last Updated: May 10, 2025

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents
07:42

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents

Published on: August 2, 2018

13.4K
Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents
07:05

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Published on: September 10, 2018

5.9K
Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
06:37

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

2.4K

Area of Science:

  • Artificial Intelligence
  • Human-Computer Interaction
  • Machine Learning

Background:

  • Learning to Defer (L2D) algorithms are vital for human-AI collaboration in high-stakes domains like fraud detection.
  • Current L2D benchmarks often use simplified simulated experts due to the high cost of real expert data.
  • This limits the realistic evaluation of L2D systems in critical applications.

Purpose of the Study:

  • To introduce OpenL2D, a novel framework for generating synthetic experts with adjustable parameters for L2D system evaluation.
  • To create a more realistic benchmark dataset for L2D algorithms using synthetic experts.
  • To analyze the impact of expert diversity on L2D algorithm performance.

Main Methods:

  • Developed OpenL2D to generate synthetic experts with controllable decision-making processes and work capacity.
  • Applied OpenL2D to a public fraud detection dataset to create the Financial Fraud Alert Review (FiFAR) dataset.
  • Collected predictions from 50 fraud analysts on 30,000 instances within the FiFAR dataset.
  • Evaluated the similarity of synthetic experts to real experts using metrics like consistency and inter-expert agreement.

Main Results:

  • The synthetic experts generated by OpenL2D demonstrated comparable consistency and inter-expert agreement to real human experts.
  • Performance rankings of different L2D algorithms varied significantly when evaluated with diverse synthetic expert pools.
  • The study highlights the critical influence of expert characteristics on L2D algorithm effectiveness.

Conclusions:

  • OpenL2D provides a scalable and realistic approach for benchmarking L2D algorithms.
  • Realistic expert modeling is essential for accurately assessing L2D system performance in real-world scenarios.
  • Future L2D research and development should account for the variability and diversity of human expert behavior.