基于Lambda架构的大数据系统的性能分析,用于使用ADS-B数据的空中/地面监视应用程序
Mustafa Umut Demirezen1, Tuğba Selcen Navruz2
1Data Products Department, UDemy Inc., San Francisco, CA 94107, USA.
Sensors (Basel, Switzerland)
|September 9, 2023
概括
这项研究通过一种新的方法验证了Lambda架构的数据处理准确性. 它经验证实了系统的最终一致性,即使在严重的数据摄入场景下,也能达到100%的准确性.
科学领域:
- 计算机科学 计算机科学
- 数据工程数据工程
- 大数据分析大数据分析
背景情况:
- 之前对Lambda架构 (LA) 的研究主要集中在硬件性能和可扩展性上.
- 从历史上看,LA的复杂设计限制了对其数据处理准确性的实证检查.
- 关于验证的方法来评估LA的准确性,在文献中存在很大的差距.
研究的目的:
- 在Lambda架构中引入和验证一种用于评估数据处理准确性的新方法.
- 提供关于LA的性能理论断言的经验证据.
- 解决超越硬件和可扩展性的以准确度为中心的评估需求.
主要方法:
- 开发了一种方法来评估所有LA层的潜在技术.
- 检查了特定层的设计限制,并实施了统一的软件开发框架.
- 利用独特的指标,包括在各种条件下数据延迟和处理精度.
主要成果:
- 经验证明了LA在速度层 (SL) 和批量层 (BL) 中的"最终一致性".
- 证实SL中的短暂不一致性得到解决,导致BL的精确结果.
- 在各种严重的数据摄入场景下实现了100%的准确率,验证了理论主张.
结论:
- 这种新的方法提供了对Lambda架构数据处理准确性的经验验证.
- LA表现出最终的一致性,确保可靠的结果,尽管实时处理波动.
- 该研究为经验支持的大数据架构评估树立了先例,适用于其他框架.
相关概念视频
Analyte Adsorption and Distribution
692
In certain chromatographic separations, solutes transfer between the mobile phase and the stationary phase via sorption, which typically refers to the process of adsorption. For many chromatographic systems, the sorption process often depends on the polarity of the compounds—an expression of the overall dipole moment within the molecule. During the separation process, there is competition between the solute and solvent for adsorption to the stationary phase. Highly polar compounds and...
692
Statistical Analysis System (SAS)
211
SAS, short for Statistical Analysis System, is a powerful data analysis, management, and visualization tool. Developed by the SAS Institute in the early 1970s, SAS has evolved into a comprehensive software suite used across various industries for statistical analysis, business intelligence, and predictive modeling.
Applications: SAS finds applications in numerous fields, including healthcare for clinical trial analysis, finance for risk assessment, marketing for customer data analysis, and...
Applications: SAS finds applications in numerous fields, including healthcare for clinical trial analysis, finance for risk assessment, marketing for customer data analysis, and...
211
Inductively Coupled Plasma Atomic Emission Spectroscopy: Instrumentation
253
Inductively coupled plasma (ICP) is the common plasma source used in atomic emission spectroscopy (AES), a technique that detects and analyzes various elements in a sample. This method is often called inductively coupled plasma atomic emission spectroscopy (ICP-AES).
There are three main types of inductively coupled plasma atomic emission spectroscopy (ICP-AES) instruments: sequential, simultaneous multichannel, and Fourier transform instruments, with the latter being less commonly used....
There are three main types of inductively coupled plasma atomic emission spectroscopy (ICP-AES) instruments: sequential, simultaneous multichannel, and Fourier transform instruments, with the latter being less commonly used....
253


