卡尔曼过器与冲动噪声异常值:一个强大的序列算法来过数据与大量异常值
Bertrand Cloez1, Bénédicte Fontez1, Eliel González-García2
1MISTEA, Univ Montpellier, INRAE, Institut Agro, Montpellier, France.
The international journal of biostatistics
|April 16, 2024
概括
本研究引入了一种新的模型和算法,用于识别数据中的冲动噪声异常值,当异常值丰富时尤其有效. 这种新方法为连续测量提供了强大而高效的数据过和异常值检测.
科学领域:
- 信号处理 信号处理
- 数据分析 数据分析
- 统计建模 统计建模
背景情况:
- 冲动噪声异常值是数据点明显偏离观测.
- 像局部回归和卡尔曼过器这样的传统方法在高异常值比例方面扎.
- 当异常值接近名义测量的密度时,现有技术是不够的.
研究的目的:
- 为冲动噪声异常值提出一个新的层次模型.
- 开发一个快速前后算法用于过,平滑和异常值检测.
- 在异常值百分比较高的场景中解决当前方法的局限性.
主要方法:
- 一个等级模型与线性高斯过程相结合,类似于卡尔曼波器.
- 一个快速前向后向的算法,用于连续的数据处理.
- 应用一个预期最大化 (EM) 算法进行参数校准.
主要成果:
- 拟议的算法有效地过和平滑序列数据,同时检测异常值.
- 与传统的异常值检测方法相比,证明了稳定性和效率.
- 在现实数据集上成功应用,其中约有60%的异常值来自Walk Over Weighing系统.
结论:
- 新的模型和算法为冲动噪声异常值检测提供了强大的解决方案,即使具有高异常频率.
- 该方法对处理顺序数据高效,适用于现实世界的场景.
- 进一步开发EM算法有助于参数校准以提高性能.
更多相关视频
06:45Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator
Published on: October 28, 2022
1.6K
05:26Author Spotlight: A Streamlined and Accessible Analysis Method to Quantify Optokinetic Reflex Tracking Responses
Published on: April 12, 2024
718
相关概念视频
Sampling Continuous Time Signal
237
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
237
Linear time-invariant Systems
253
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
253
Quantifying and Rejecting Outliers: The Grubbs Test
1.6K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.6K
Linear Approximation in Frequency Domain
89
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
89
Linear Approximation in Time Domain
81
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
81
Deconvolution
159
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
159
