数据管道用于实时能源消耗数据管理和预测
Jeonghwan Im1, Jaekyu Lee1, Somin Lee2
1Graduate School of Data Science, Seoul National University of Science and Technology, Seoul, Republic of Korea.
Frontiers in big data
|April 2, 2024
概括
本研究介绍了用于能源管理的机器学习操作中心数据管道. 拟议的系统提供高效的实时数据处理和预测,显著优于现有方法.
科学领域:
- 数据工程数据工程
- 机器学习操作 (MLOps)
- 能源系统 能源系统
背景情况:
- 有效的数据管道对于跨行业的实时数据利用至关重要.
- 能源消耗管理系统需要强大的数据处理和预测能力.
研究的目的:
- 为能源消耗管理提出一个以机器学习作业为中心的数据管道.
- 将实时数据管理和预测与机器学习模型集成在一起.
- 评估管道的效率和可扩展性.
主要方法:
- 使用Kafka,InfluxDB,Telegraf,Zookeeper和Grafana开发了一个数据管道架构.
- 实施并比较了两个时间序列预测模型:长期短期记忆 (LSTM) 和季节性自回归集成移动平均值 (SARIMA).
- 优化了 Telegraf 配置以测量端到端处理时间.
主要成果:
- 管道实现了平均端到端处理时间为0.39s的10,000条记录和1.26s的10万条记录.
- 演示的处理速度比现有的基于Python的方法快30.69-90.88倍.
- 当数据量增加了十倍时,增加的开销减少了3.07倍.
- 确定了速度与准确性的权衡:SARIMA更快,而LSTM提供更高的预测准确性.
结论:
- 拟议的管道是高效和可扩展的实时能源消耗管理.
- 该架构有效地整合了数据管理,预测和MLOps原则.
- 该系统比传统方法提供了显著的性能改进.
相关概念视频
01:17Energy and Power Signals
287
In an electrical system with a resistor, voltage and current signals facilitate the measurement of power and energy across the resistor. For a continuous-time signal, the total energy over a time interval is defined as the integral of the square of the signal's magnitude over that interval. Mathematically, this is expressed as:
287
01:12Run Charts
59
Run charts serve as an essential instrument for visualizing the performance of various processes over time, enabling the identification of trends and patterns crucial for quality improvement. These charts map out a series of data points chronologically, offering insights into the stability and efficiency of a process. A run chart's creation involves plotting data points on a graph, with the time intervals on the horizontal axis and the specific measurements on the vertical axis. For...
59
01:25Distribution Reliability and Automation
107
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
107
01:10Electrical Energy
1.2K
Using electric appliances for a longer period of time consumes more electrical energy and results in a higher electric bill. The energy produced by the transfer of electrons from one point to another is known as electrical energy. If power is delivered at a constant rate, the electrical energy can be defined as the product of power used by the device for a period of time. The energy unit on electric bills is the kilowatt-hour, where one kilowatt-hour is equivalent to 3.6 × 106 joules.
1.2K
01:26The Power Flow Problem and Solution
211
Power flow problem analysis is fundamental for determining real and reactive power flows in network components, such as transmission lines, transformers, and loads. The power system's single-line diagram provides data on the bus, transmission line, and transformer. Each bus k in the system is characterized by four key variables: voltage magnitude Vk, phase angle δk, real power Pk, and reactive power Qk. Two of these four variables are inputs, while the...
211
01:24Fast Decoupled and DC Powerflow
191
The fast decoupled power flow method addresses contingencies in power system operations, such as generator outages or transmission line failures. This method provides quick power flow solutions, essential for real-time system adjustments. Fast decoupled power flow algorithms simplify the Jacobian matrix by neglecting certain elements, leading to two sets of decoupled equations:
191