视频A11y:可访问视频描述的方法和数据集
Chaoyu Li1, Sid Padmanabhuni1, Maryam S Cheema1
1School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA.
概括
视频A11y使用多式大型语言模型 (MLLM) 来为盲人和视力低下者 (BLV) 创建更好的视频描述. 与目前的方法相比,这种方法显著提高了可访问性和用户满意度.
科学领域:
- 计算机科学
- 人与计算机的交互
- 人工智能
背景情况:
- 视频描述对于盲人和视力低下者 (BLV) 的可访问性至关重要.
- 由于训练数据的局限性,现有的人工智能描述往往缺乏质量,无法满足BLV用户的需求.
研究的目的:
- 用多式大型语言模型 (MLLM) 和可访问性准则为BLV用户生成高质量的视频描述,开发一种方法 (VideoA11y).
- 创建视频A11y-40K数据集,这是BLV用户最大的描述视频集合.
主要方法:
- 利用MLLM和视频可访问性准则来生成描述.
- 编辑包括4万个描述视频的VideoA11y-40K数据集.
- 对视觉和BLV参与者以及专业描述者进行实验以评估描述质量.
主要成果:
- 发现视频A11y的描述优于新手的人类注释.
- 产生的描述在清晰度,准确性,客观性,描述性和用户满意度方面表现出与受过训练的人类注释相当的质量.
- 在VideoA11y-40K数据集上微调的MLLM产生了高质量的可访问的视频描述.
结论:
- 视频A11y方法有效地为BLV用户生成可访问的视频描述.
- 视频A11y-40K数据集和MLLM微调可以提高视频可访问性.
- 这项工作提供了一个可扩展的解决方案,
更多相关视频
10:11Portable Intermodal Preferential Looking IPL: Investigating Language Comprehension in Typically Developing Toddlers and Young Children with Autism
Published on: December 14, 2012
18.5K
08:25Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
Published on: May 7, 2019
9.1K
相关概念视频
Data Collection by Observations
Data collection refers to a systematic way of obtaining, observing, measuring, and analyzing accurate information. Observational studies are one of the most widely used methods of data collection. It involves collecting data by observing the behavior and physical characteristics of a sample without making any modifications to the sample.
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
Data Collection I
Data collection gathers information needed to make accurate judgments about a patient's present condition. During a health history interview, subjective data is collected from the patient, their caregivers, or family members, and objective data is collected through observations and physical assessment. Patients are the primary source of subjective data. Thus information gathered from patients through interviews, observations, and physical examination is primary data. Secondary sources of data...
Data Collection II
The nursing history captures and records the patient's health status, so that a care plan evolves to meet the patient's individual needs. The nursing health history is a part of the initial assessment. A comprehensive history covers all health dimensions and plays a significant role in the assessment process. A comprehensive history includes the patient's biographical information, reasons for seeking health care, expectations, present and past health history, medications, and family,...
Data Reporting and Recording
Reporting and recording are crucial in data documentation. The timely, thorough, and accurate documentation of facts is essential when recording patient data. Failure to record findings during an assessment or interpretation of a problem will result in loss of information and make the patient document unreliable. The reader is left with general impressions if the information is not specific. A recording is documenting data of the individual's health information in a traceable, secure, and...
