通过基于SSD的Hadoop mapreduce提高电子政府信息系统的性能
1Department of Information Systems and Technology, College of Informatics and Virtual Education, The University of Dodoma, Dodoma, Tanzania. ishengomaf@gmail.com.
Scientific reports
|October 1, 2025
概括
这项研究引入了Hadoop集群与固态驱动器 (SSD) 的新混合机制,显著提高了电子政府数据处理速度. 最优化的方法减少了I / O和网络延迟,提高了整体系统效率.
科学领域:
- 计算机科学 计算机科学
- 数据工程数据工程
- 电子政府 (E-Government) 是一个电子政府.
背景情况:
- 电子政府系统产生了庞大的,异质的数据集,需要高吞吐量,低延迟处理.
- 虽然Hadoop MapReduce很常见,但由于盘 I/O 和混合阶段的网络延迟而面临性能瓶.
研究的目的:
- 为使用固态驱动器 (SSD) 的 Hadoop 集群提出和评估基于数据地址的混合机制.
- 为了提高大规模电子政府应用的数据处理性能和效率.
主要方法:
- 实施一种新的混合机制,包括基于地址的分类,合并和中间数据的预发送.
- 在配备SSD的Hadoop集群上使用Terasort和Wordcount基准进行性能评估.
- 在模拟的50节点集群上进行可扩展性测试和能源消耗概况.
主要成果:
- 实现了8% (Terasort) 和1% (Wordcount) 的执行时间缩短,具有统计学上显著的95%置信区间.
- 与硬盘驱动器 (HDD) 系统相比,表现出更好的性能,减少了网络拥堵,能源消耗降低了31%.
- 验证了大规模数据处理的增强效率和可扩展性.
结论:
- 拟议的基于数据地址的混合机制为公共部门计算中的大数据处理提供了具有成本效益和高效的解决方案.
- 优化SSD的混合机制对于推进电子政府数据处理能力至关重要.
- 这种方法有效地减轻了I / O和网络开销,从而带来了显著的性能提升.
相关概念视频
01:30Health Information Technology and Healthcare Information System
3.3K
Health Information Technology (HIT)
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
3.3K
01:29Applications of GIS: Disaster Management and Emergency Response
471
Geographic Information System (GIS) technology is essential for risk identification, action prioritization, and resource optimization in critical situations like flooding and earthquakes. By integrating spatial and demographic data, GIS provides a comprehensive framework for emergency response.GIS integrates data layers, like rainfall intensity, topography, elevation profiles, and river levels, to model high-risk flood zones. These layers assess areas susceptible to flooding based on their...
471
01:28Introduction to GIS
514
Geographic Information Systems (GIS) are tools for storing, analyzing, and displaying spatial data alongside related attributes. Unlike traditional information systems that address general queries, GIS incorporates spatial components, enabling users to answer "where" and "how far." For example, GIS can process housing data linked to geographic locations like zip codes, allowing insights into population density or housing distribution through thematic maps.GIS integrates technologies such as...
514
01:29Levels of Use of a GIS
358
Geographic Information Systems (GIS) operate across three levels of application, each representing an increasing degree of complexity: data management, analysis, and prediction. These levels reflect the expanding functionality and versatility of GIS technology in handling spatial data for diverse purposes.Data ManagementAt its foundational level, GIS serves as a tool for data management, enabling the input, storage, retrieval, and organization of spatial data. This level is often employed in...
358
01:21Manipulation and Analysis
287
GIS manipulation and analysis functions are vital for decision-making and planning. These activities range from data retrieval tasks, such as selecting information based on specific criteria, to advanced analytical techniques that address complex spatial problems.One critical GIS analysis method is overlaying, which combines multiple data layers to examine impacts. For example, overlaying a river-dammed lake boundary with road networks can identify affected infrastructure. Another common...
287
01:30Thematic Layering in GIS
332
In the past, planning projects such as schools or public facilities required extensive manual effort to gather and compile data. Information such as property boundaries, soil characteristics, road networks, zoning regulations, and flood zones had to be sourced individually from courthouses, utility providers, and registry offices. Assembling these datasets into a coherent format often took several months, delaying project timelines.The introduction of Geographic Information Systems (GIS)...
332