在线标签聚合与不完整的群众响应
Yuyang Liu1,2, Haoyu Liu2, Runze Wu3
1Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
概括
本研究介绍了OLA-Incomplete,这是一个新的在线标签聚合框架,用于处理异步和不完整的众包数据. 它通过整合生成重复和变异推理来实现高精度,优于现有方法.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 数据科学数据科学数据科学
背景情况:
- 传统的众包聚合方法与异步和不完整的响应流扎.
- 现有的在线方法通常需要每个步骤完整的数据或低效地重新加载历史响应,这给存储和隐私带来了挑战.
研究的目的:
- 开发一个在线标签聚合框架,OLA-Incomplete,专门设计用于众包中不完整的响应流.
- 通过减轻灾难性遗忘和模拟未知工人的可靠性来解决先前方法的局限性.
主要方法:
- 引入了OLA-Incomplete,将一个变异推理聚合器与一个生成重复模块集成在一起.
- 生成重播模块保存历史信息而无需重新加载,通过重播过去的数据来缓解遗忘.
- 聚合器通过将证据最大化来推断真相,通过重复和新标签来最大限度地限制证据,明确地建模工人的可靠性.
主要成果:
- 在公共数据集上实现了高最终准确率:90.74% (Duck),92.50% (RTE) 和95.99% (PostSent).
- 与最强的基线相比,至少表现出7.79%的相对改善.
- 展示了强大的即时在线准确性和稳定性,以适应不同的响应块大小和到达订单.
结论:
- OLA-Incomplete提供了一个实用和有效的解决方案,用于现实世界的众包工作流与不完整的数据.
- 该框架能够处理异步,不完整的响应和模型工作者的可靠性,提高了数据聚合的准确性和效率.
相关概念视频
Surveys
17.1K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
17.1K
Data Collection by Survey
9.4K
The systematic method of obtaining and analyzing accurate information of a population is called data collection. A survey is a standard method of data collection that involves collecting information from a target human population about their experience, opinion, or knowledge of a product, service, or process. The responses are recorded and interpreted. The most common survey examples are written questionnaires, face-to-face or telephonic conversations, focus groups, and electronic (e-mail or...
9.4K
Data Collection by Observations
15.4K
Data collection refers to a systematic way of obtaining, observing, measuring, and analyzing accurate information. Observational studies are one of the most widely used methods of data collection. It involves collecting data by observing the behavior and physical characteristics of a sample without making any modifications to the sample.
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
15.4K
Data Collection I
8.8K
Data collection gathers information needed to make accurate judgments about a patient's present condition. During a health history interview, subjective data is collected from the patient, their caregivers, or family members, and objective data is collected through observations and physical assessment. Patients are the primary source of subjective data. Thus information gathered from patients through interviews, observations, and physical examination is primary data. Secondary sources of...
8.8K
Cluster Sampling Method
15.2K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
15.2K
Data Collection III
4.7K
The physical assessment examines the patient for objective data that defines the patient's condition, and aids in formulating the nursing care plan. The purpose of physical assessment is a health status appraisal, which includes identifying health problems, and establishing a database for nursing intervention.
The principles to begin the physical assessment include conducting a comprehensive or problem-related history in a quiet, well-lit room, emphasizing privacy and comfort for the...
The principles to begin the physical assessment include conducting a comprehensive or problem-related history in a quiet, well-lit room, emphasizing privacy and comfort for the...
4.7K


