对大规模数据链接的超采样-低采样策略
Hossein Hassani1, Mohammad Reza Entezarian2, Sara Zaeimzadeh3
1International Institute for Applied Systems Analysis (IIASA), Laxenburg, Austria.
Frontiers in big data
|May 8, 2025
概括
本研究引入了一种过量采样-不足采样策略,以提高不平衡的大数据中记录链接的准确性. 通过平衡数据集,它提高了大规模信息系统中链接记录的效率.
科学领域:
- 数据科学数据科学数据科学
- 计算机科学 计算机科学
- 信息科学 信息科学 信息科学
背景情况:
- 有效的记录链接对于大数据分析至关重要,但在不平衡的数据集中具有挑战性.
- 不平衡的数据集,其中一个类明显超过其他类,妨碍准确的记录链接.
- 现有的方法难以应对大规模,不平衡数据的复杂性.
研究的目的:
- 为记录链接中不平衡的数据集制定和评估一个过量采样-过少采样策略.
- 提高大数据环境中记录链接的准确性和效率.
- 解决大规模数据链接任务中阶级不平衡所带来的挑战.
主要方法:
- 实施过量抽样-不足抽样技术来平衡数据集.
- 调整了少数阶级和多数阶级的实例数.
- 通过不同的训练测试比率和不平衡度进行了敏感性测试.
主要成果:
- 过量抽样-过少抽样策略有效地平衡了数据集.
- 记录链接的准确性和效率有所提高.
- 灵敏度分析提供了对不同条件下的方法稳定性的见解.
结论:
- 提议的过量抽样-不足抽样策略是有效的改善记录链接在不平衡的大数据.
- 通过这种方法平衡数据集,可以实现更准确,更有效的记录链接.
- 该方法为处理具有显著类失衡的大规模数据集提供了可行的解决方案.
相关概念视频
Stratified Sampling Method
11.6K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a stratified sample, divide the population into groups called strata and then take a...
To choose a stratified sample, divide the population into groups called strata and then take a...
11.6K
Sampling Plans
155
Sampling is a crucial step in analytical chemistry, allowing researchers to collect representative data from a large population. Common sampling methods include random, judgmental, systematic, stratified, and cluster sampling.
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
155
Cluster Sampling Method
11.5K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
11.5K
Systematic Sampling Method
9.9K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
Systematic sampling is one of the simplest methods...
Systematic sampling is one of the simplest methods...
9.9K
Sampling Methods: Overview
236
A sample refers to a smaller subset representative of a larger population. In analytical chemistry, studying or analyzing an entire population is often impractical or impossible. Therefore, samples are used to draw inferences and generalize the whole population. The sampling method selects individuals or items from a population to create a sample. Standard sampling methods include random, judgemental, systematic, stratified, and cluster sampling.
In analytical chemistry, the choice of...
In analytical chemistry, the choice of...
236
Convenience Sampling Method
8.5K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population.
Convenience sampling is a non-random method of sample selection; this method selects individuals that are easily accessible and may result in biased data. For example, a marketing...
Convenience sampling is a non-random method of sample selection; this method selects individuals that are easily accessible and may result in biased data. For example, a marketing...
8.5K


