基于过和文本增强的跨语言总结的数据集构建方法
Hangyu Pan1, Yaoyi Xi1, Ling Wang1
1State Key Laboratory of Mathematical Engineering and Advanced Computing, Zhengzhou, China.
PeerJ. Computer science
|June 22, 2023
概括
我们开发了一种新的方法来创建高质量的,大规模的跨语言总结 (CLS) 数据集. 这种方法使用过和文本增强来有效地提高样本质量和数据集大小.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
- 数据科学数据科学数据科学
背景情况:
- 现有的跨语言总结 (CLS) 数据集存在不一致的样本质量和有限的规模.
- 这阻碍了有效的CLS模型的开发和评估.
研究的目的:
- 提出一种新的方法,共同监督CLS数据集构建中的质量和规模.
- 通过提高数据质量和大小来解决现有的CLS数据集的局限性.
主要方法:
- 实施了多策略过算法,以根据字符和语义分析删除低质量的单语言总结 (MS) 样本.
- 利用预训练模型利用文本增强算法来扩大CLS数据集大小,同时确保质量.
主要成果:
- 使用提出的方法成功构建了一个英语-中文CLS数据集.
- 使用强大的质量评估框架评估数据集,确认质量好,规模大.
- 证明该方法以更低的成本全面提高质量和规模.
结论:
- 拟议的方法提供了一种有效的方法,用于构建高质量,大规模的CLS数据集.
- 这有助于推进跨语言总结的研究和开发.
- 该方法为数据集创建提供了具有成本效益的解决方案.
相关概念视频
02:07Improving Translational Accuracy
2.6K
2.6K
01:20Cluster Sampling Method
12.0K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
12.0K
01:50Cross-Sectional Research
11.4K
In cross-sectional research, a researcher compares multiple segments of the population at the same time. If they were interested in people's dietary habits, the researcher might directly compare different groups of people by age. Instead of following a group of people for 20 years to see how their dietary habits changed from decade to decade, the researcher would study a group of 20-year-old individuals and compare them to a group of 30-year-old individuals and a group of 40-year-old...
11.4K
01:16Stratified Sampling Method
12.1K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a stratified sample, divide the population into groups called strata and then take a...
To choose a stratified sample, divide the population into groups called strata and then take a...
12.1K
01:09Random Sampling Method
11.2K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest. Among the various sampling methods used by...
11.2K
01:07Data Collection by Survey
6.5K
The systematic method of obtaining and analyzing accurate information of a population is called data collection. A survey is a standard method of data collection that involves collecting information from a target human population about their experience, opinion, or knowledge of a product, service, or process. The responses are recorded and interpreted. The most common survey examples are written questionnaires, face-to-face or telephonic conversations, focus groups, and electronic (e-mail or...
6.5K