Distance-Entropy: An Effective Indicator for Selecting Informative Data
Yang Li1,2, Xuewei Chao1
1College of Mechanical and Electrical Engineering, Shihezi University, Xinjiang, China.
Frontiers in Plant Science
|January 31, 2022
Summary
This study introduces distance-entropy, a novel indicator for assessing agricultural data quality based on information content, not just visual appeal. This method enhances smart agriculture by improving data accuracy for pest recognition.
Area of Science:
- Agricultural Science
- Data Science
- Computer Vision
Background:
- Smart agriculture relies heavily on data for efficiency and cost reduction.
- Current image quality assessment overlooks information content, focusing only on visual aspects.
- High-quality data is essential for effective intelligent algorithms in agriculture.
Purpose of the Study:
- To propose and validate a new indicator, distance-entropy, for assessing agricultural data quality from an information perspective.
- To address the limitations of existing visual quality assessments in smart agriculture.
- To improve the reliability of data used in crop pest recognition tasks.
Main Methods:
- Developed the distance-entropy indicator to differentiate data quality based on information content.
- Conducted comparative experiments varying mapping feature dimensions and base data sizes.
- Validated the indicator's effectiveness and robustness using numerical and visual results.
Main Results:
- The distance-entropy method effectively distinguished between good and bad data.
- Experimental results demonstrated the indicator's validity and robustness across different parameters.
- Both quantitative and qualitative analyses confirmed the method's stability and effectiveness.
Conclusions:
- The proposed distance-entropy indicator offers a novel approach to data quality assessment in smart agriculture.
- This work highlights the importance of information quality in agricultural datasets.
- Findings provide inspiration for data mining and dataset optimization in practical agricultural applications.
Related Concept Videos
01:11Statistical Analysis: Overview
8.8K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
8.8K
01:08Outliers and Influential Points
4.7K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.7K
01:00Detection of Gross Error: The Q Test
6.5K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.5K
01:12What Are Outliers?
4.4K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
4.4K
01:20Entropy and the Second Law of Thermodynamics
3.3K
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
3.3K
01:02Quantifying and Rejecting Outliers: The Grubbs Test
2.5K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.5K