基于稀疏的自我编码的异常能量大数据的清理方法
Dongge Zhu1, Shuang Zhang1, Rui Ma2
1Electric Power Research Institute of State Grid Ningxia Electric Power Co., Ltd. Yinchuan, Ningxia, 750002, China.
Scientific reports
|October 14, 2024
概括
这项研究引入了一种新的稀疏自编码方法来清理异常能量大数据,显著提高异常检测的准确性. 这种方法有效地减少了数据干扰,以最小的处理时间实现高清理速率.
科学领域:
- 数据科学数据科学数据科学
- 能源系统分析 能源系统分析
- 机器学习 机器学习
背景情况:
- 异常能量大数据对准确的分析和异常检测提出了挑战.
- 现有的方法与异常数据的干扰作斗争,影响可靠性.
- 动态碳排放影响能源数据,需要先进的清洁技术.
研究的目的:
- 提出和验证一种稀疏的自我编码方法来清理异常能量大数据.
- 为了提高异常检测和数据清理过程的准确性.
- 解决能源大数据库中异常数据的干扰问题.
主要方法:
- 对异常数据检测和光谱特征分析的多标准评估.
- 混乱的时间序列重建,强大的局部加权回归和稀疏的自我编码来进行特征分解.
- 基于周期性的自适应细分,异常指数的AFCM算法和基于LOF的评估模型.
主要成果:
- 实现了0.24%的错误检测率和0.27%的缺失检测率.
- 对于异常能量大数据,证明了99.49%的清洁率.
- 在不到2秒的时间内完成数据清理,展示了高效率.
结论:
- 建议的稀疏自编码方法对于清理异常能量大数据非常有效.
- 该方法显著提高了异常检测的准确性,并减少了数据干扰.
- 该方法为现实世界能源数据清理应用提供了快速有效的解决方案.
相关概念视频
01:20Cluster Sampling Method
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
01:28Methods of Medium Optimization
Optimizing growth media enhances microbial proliferation and maximizes product yield. Statistical experimental design methodologies provide structured and reproducible approaches, offering progressively higher levels of robustness and efficiency.The One-Factor-at-a-Time (OFAT) MethodThe One-Factor-at-a-Time (OFAT) method involves adjusting a single variable while keeping all others constant. However, it cannot detect interactions between variables, often leading to suboptimal outcomes when...