孟加拉语手语:一种基于机器学习的孟加拉语手语口译,用于聋人和非口语人士
Md Johir Raihan1, Mainul Islam Labib1, Abdullah Al Jaid Jim1,2
1Electronics and Communication Engineering Discipline, Khulna University, Khulna 9208, Bangladesh.
Sensors (Basel, Switzerland)
|August 29, 2024
概括
这项研究通过开发一个卷积神经网络-挤压激发网络来弥合聋人和听觉个体之间的沟通差距,以准确识别手语. 一个智能手机应用程序使这个先进的机器学习模型可供所有人使用.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 人与计算机的交互
背景情况:
- 手语是聋人的主要沟通方式,但它的使用在听力人口中是有限的.
- 聋人和听力社区之间仍然存在很大的沟通差异.
- 弥合这一差距对于促进包容性和相互理解至关重要.
研究的目的:
- 开发一个准确的机器学习模型用于手语识别.
- 创建一个用户友好的智能手机应用程序,以访问手语识别模型.
- 提高机器学习模型预测的可解释性.
主要方法:
- 一个卷积神经网络-挤压激发网络被开发用于手语信号预测.
- 创建了一个智能手机应用程序来部署机器学习模型.
- 为了模型的解释性,使用了沙普利增量解释法 (SHAP).
主要成果:
- 开发的模型在KU-BdSL数据集上实现了99.86%的高精度.
- 网络中的挤压激发 (SE) 块通过专注于相关图像通道来提高模型性能.
- SHAP分析证实该模型依赖于与手有关的视觉线索,反映了人类手语解释.
结论:
- 该研究成功开发了一种准确且易于使用的手语识别系统.
- 集成的SE块和SHAP分析提供了一个强大的方法来改进和理解机器学习模型在这个领域.
- 智能手机应用程序使手语技术的访问变得民主化,促进聋人和听觉人士之间更好的沟通.
更多相关视频
06:07Exploring Infant Sensitivity to Visual Language using Eye Tracking and the Preferential Looking Paradigm
Published on: May 15, 2019
8.3K
11:39Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
Published on: September 7, 2022
2.1K
相关概念视频
Genetic Lingo
Overview
Translation
Lesson: Translation
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of Life
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of Life
Translation
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of Life
Proteins are called the...
Translation Produces the Building Blocks of Life
Proteins are called the...
Language
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Components of Language
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs. “eh”). Phonemes combine to...
Non-Verbal Cues
Non-verbal communication extends beyond gestures and facial expressions to include vocal elements known as paralanguage. Paralanguage consists of non-verbal vocal cues such as pitch, loudness, speech rate, pauses, and non-verbal vocalizations like laughter, sighs, and moans. These elements not only accompany speech but also provide critical emotional and contextual information.The Role of Paralanguage in CommunicationParalanguage adds depth to spoken language by conveying emotions and...
