Related Experiment Video
Updated: Oct 23, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
774
Top-k dominating queries on incomplete large dataset.
Jimmy Ming-Tai Wu1, Min Wei1, Mu-En Wu2
1College of Computer Science and Engineering, Shandong University of Science and Technology, Qindao, China.
The Journal of Supercomputing
|August 23, 2021
Summary
This study introduces an Efficient Hadoop BitMap Index Guided Algorithm (EHBIG) for top-k dominating (TKD) queries on large, incomplete datasets. The EHBIG algorithm, along with its improved version (IEHBIG), significantly enhances performance and reduces resource usage.
Area of Science:
- Data Mining
- Database Systems
- Big Data Analytics
Background:
- Top-k dominating (TKD) queries identify objects that dominate others in a dataset.
- Incomplete datasets with missing values pose challenges for traditional data mining.
- Existing methods struggle with TKD queries on large-scale incomplete data.
Purpose of the Study:
- To develop an efficient algorithm for TKD queries on large incomplete datasets.
- To leverage MapReduce architecture for scalability and performance.
- To reduce runtime and memory usage through a pruning strategy.
Main Methods:
- Proposed the Efficient Hadoop BitMap Index Guided Algorithm (EHBIG) using MapReduce and BitMap Index.
- Implemented a pruning strategy to optimize query processing.
- Developed an improved version, IEHBIG, for enhanced algorithm flow.
Main Results:
- EHBIG successfully performs TKD queries on incomplete large datasets.
- The pruning strategy significantly reduces runtime and memory consumption.
- Experimental results demonstrate strong performance in a Hadoop cluster.
Conclusions:
- EHBIG and IEHBIG provide an effective solution for TKD queries on big incomplete data.
- The MapReduce architecture ensures scalability for large datasets.
- The proposed algorithms offer significant performance improvements over existing methods.
Related Concept Videos
01:43Incomplete Dominance
28.0K
Gregor Mendel's work (1822 - 1884) was primarily focused on pea plants. Through his initial experiments, he determined that every gene in a diploid cell has two variants called alleles inherited from each parent. He suggested that amongst these two alleles, one allele is dominant in character and the other recessive. The combination of alleles determines the phenotype of a gene in an organism.
28.0K
01:12Maximum Size of Aggregate
288
The maximum size of aggregate is defined as the aperture of the sieve retaining 15 percent or more of the particles present in the aggregate sample. The aggregate's maximum size impacts the concrete's water requirement, workability, and strength. Larger aggregates reduce the surface area needing cement paste coverage, which can lower water needs, thereby allowing a decrease in the water-to-cement ratio when the desired workability and richness of the mix are to be maintained, which can...
288
01:12Column Efficiency: Rate Theory
608
The rate theory of chromatography provides quantitative insight into the shapes and widths of elution bands. These bands are based on the random-walk mechanism governing molecular migration within a column. The Gaussian profile of chromatographic bands arises from the cumulative effect of random molecular motions as they progress through the column.
During elution, a solute molecule experiences numerous transitions between stationary and mobile phases, exhibiting irregular residence times in...
During elution, a solute molecule experiences numerous transitions between stationary and mobile phases, exhibiting irregular residence times in...
608
01:19The Mantel-Cox Log-Rank Test
671
The Mantel-Cox log-rank test is a widely used statistical method for comparing the survival distributions of two groups. It tests whether a statistically significant difference exists in survival times between the groups without assuming a specific distribution for the survival data, making it a non-parametric test. This flexibility makes the log-rank test particularly valuable in medical research and other fields where the timing of an event, such as death or disease recurrence, is of...
671
01:20Maxwell-Boltzmann Distribution: Problem Solving
1.9K
Individual molecules in a gas move in random directions, but a gas containing numerous molecules has a predictable distribution of molecular speeds, which is known as the Maxwell-Boltzmann distribution, f(v).
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
1.9K
01:17Wald-Wolfowitz Runs Test I
763
The Wald-Wolfowitz test, also known as the runs test, is a nonparametric statistical test used to assess the randomness of a sequence of two different types of elements (e.g., positive/negative values, successes/failures). It examines whether the order of the elements in a sequence is random or if there is a pattern or trend present. This nonparametric test applies to any ordered data despite the population and sample data distribution, even if a higher sample size is available.
The test works...
The test works...
763