Related Experiment Video
Updated: Dec 28, 2025
09:41
Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
Published on: April 21, 2023
2.1K
Hands Holding Clues for Object Recognition in Teachable Machines
Kyungjun Lee1, Hernisa Kacorri2
1Department of Computer Science, University of Maryland, College Park, MD, USA.
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI Conference
|February 12, 2020
Summary
This study introduces a deep learning system to improve object recognition for blind individuals by using hand segmentation and localization. This approach helps users naturally indicate objects, enhancing performance in assistive technology.
Area of Science:
- Computer Vision
- Human-Computer Interaction
- Assistive Technology
Background:
- Camera manipulation challenges object recognition for blind users.
- Training data from visually impaired individuals can degrade model performance due to framing issues.
- Proprioception guides hand movements, offering a potential interface for object interaction.
Purpose of the Study:
- To develop a deep learning system for object recognition that leverages hand movements.
- To investigate the use of hand segmentation and object localization for improved classification.
- To enhance object recognition applications for blind and visually impaired users.
Main Methods:
- Proposed a deep learning system integrating hand segmentation and object localization.
- Utilized proprioception-based hand coordination principles.
- Analyzed existing datasets and introduced a new egocentric dataset for validation.
Main Results:
- Demonstrated the potential of using hands as a natural interface for object indication.
- Identified performance degradation factors in current object recognition models.
- Provided insights into the effectiveness of the proposed approach through error analysis.
Conclusions:
- Jointly modeling hand segmentation and object localization offers a promising solution for assistive object recognition.
- The proposed system can improve the usability of teachable machines for visually impaired users.
- Further research can refine this approach for more robust and intuitive assistive technologies.
Related Concept Videos
01:30Machines: Problem Solving II
591
Machines are complex structures consisting of movable, pin-connected multi-force members that work together to transmit forces. Consider a lifting tong carrying a 100 kg load. It comprises movable sections DAF and CBG linked together with member AB.
591
01:22Machines: Problem Solving I
631
A toggle clamp is a mechanical device commonly used for holding and clamping objects in various applications, such as woodworking, metalworking, and assembly operations. Consider a toggle clamp subjected to a force of 200 N at the handle. The vertical clamping force can be calculated, provided the dimensions of the toggle clamp are known.
The toggle clamp system is a machine structure consisting of movable, pin-connected multi-force members that form a stabilized system to transmit forces. The...
The toggle clamp system is a machine structure consisting of movable, pin-connected multi-force members that form a stabilized system to transmit forces. The...
631
01:22Force Classification
2.2K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
2.2K
01:12Observational Learning
747
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
747
01:22Purposive Learning
382
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
382
01:17Muscles of the Forearm that Move the Hand and Fingers
2.3K
The muscles of the forearm that move the wrist, hand, and digits are numerous and diverse. They can be classified into two groups based on their location and function — the anterior and posterior compartment muscles.
Anterior Compartment
The anterior compartment muscles originate from the humerus. They primarily function as flexors and are also known as flexor muscles. They typically insert on the carpals, metacarpals, and phalanges. The superficial layer includes the flexor carpi...
Anterior Compartment
The anterior compartment muscles originate from the humerus. They primarily function as flexors and are also known as flexor muscles. They typically insert on the carpals, metacarpals, and phalanges. The superficial layer includes the flexor carpi...
2.3K