增强聋人听力沟通能力:探索基于人工智能的预测和生成策略之间的协同作用,以实现 (葡萄牙语) 手语解释
Telmo Adão1,2, João Oliveira3, Somayeh Shahrabadi3
1Department of Engineering, School of Sciences and Technology, University of Trás-os-Montes e Alto Douro, 5000-801 Vila Real, Portugal.
Journal of imaging
|November 24, 2023
概括
本研究介绍了一种新的葡萄牙手语解释系统,使用LSTM网络和数据增强来提高准确性. 一种独特的缓冲技术有助于符号化,增强聋人和听觉人士的沟通包容性.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 人与计算机的交互
背景情况:
- 对于包容性而言,聋人和听力人士之间的有效沟通至关重要.
- 现有的手语识别 (SLR) 数字解决方案在跨平台兼容性和持续对话支持方面面临挑战.
- 葡萄牙手语 (LGP) 翻译需要先进的方法来实现准确和流的沟通.
研究的目的:
- 提出一个非侵入性的LGP解释系统-as-a-service.
- 通过数据集增强来增强机器学习模型训练.
- 改进手语的标记化和句子构造,以实现自然沟通.
主要方法:
- 利用长短期记忆 (LSTM) 架构来推断骨姿势序列.
- 实施数据集增强策略,以解决机器学习中的数据稀缺问题.
- 开发基于缓冲器的交互技术,用于实时LGP术语代币化.
- 整合一个大型语言模型 (LLM) 以实现类似人类的解释条件.
主要成果:
- 用50个LGP术语和数据增强训练的LSTM模型实现了80%至95.6%的准确性.
- 用户报告了基于缓冲器的代币化交互策略的高直观性.
- 在LLM整合 (ChatGPT) 中,在生成的句子中显示出有希望的语义相关率.
结论:
- 拟议的系统有效地高准确地解释葡萄牙手语.
- 基于缓冲区的交互技术显著改善了用户体验和句子的一致性.
- 整合LLM具有更自然和更具背景意识的手语解释的潜力.
关键词:
葡萄牙手语 葡萄牙手语聋人听力沟通的沟通方式产生预训练变压器 (GPT)包容性 包容性 包含性大型语言模型 (LLM)长期和短期记忆 (LSTM)机器学习 (ML) 是指机器学习.标记语言识别 (SLR) 系统基于视频的运动分析.更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
236
07:18Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication
Published on: January 26, 2024
907
相关概念视频
Improving Translational Accuracy
11.2K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.2K
Non-equilibrium in the Cell
4.4K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.4K
Multi-input and Multi-variable systems
108
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
108
Associative Learning
407
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
407
Language Development
369
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
369
Language and Cognition
352
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
352
