Related Experiment Video
Updated: Nov 21, 2025
09:44
Use of Principal Components for Scaling Up Topographic Models to Map Soil Redistribution and Soil Organic Carbon
Published on: October 16, 2018
10.5K
A Parallel Computing Approach to Spatial Neighboring Analysis of Large Amounts of Terrain Data Using Spark
Jianbo Zhang1, Zhuangzhuang Ye1, Kai Zheng1
1School of Geography Information Engineering, China University of Geosciecnes, Wuhan 430074, China.
Sensors (Basel, Switzerland)
|January 12, 2021
Summary
This study introduces a new parallel computing approach for spatial analysis on large raster datasets using Apache Spark. The method efficiently processes terrain data, improving performance and accuracy for focal statistics algorithms.
Area of Science:
- Geospatial analysis
- Computational geography
- Big data analytics
Background:
- High-resolution raster data present computational challenges for traditional focal statistics algorithms.
- Existing in-memory computing frameworks like Apache Spark may face performance issues with spatial data due to limited native support.
- Efficient processing of large-scale spatial data is crucial in the big data era.
Purpose of the Study:
- To develop a Spark-based parallel computing approach for focal algorithms in spatial neighboring analysis.
- To address the computational challenges posed by large raster datasets in geo-raster spatial analysis.
- To enhance the performance and scalability of focal statistics algorithms for big spatial data.
Main Methods:
- A tile-based multifile storing strategy partitions raster digital elevation model (DEM) files for Hadoop Distributed File System (HDFS).
- A dynamic calculation window (DCW) computing strategy is employed to perform slope algorithms on tile files.
- Results are written back and merged into a complete raster file.
Main Results:
- The proposed approach significantly improves the parallel performance of focal statistics algorithms.
- Experimental results demonstrate calculation accuracy comparable to established software like ArcGIS.
- The approach exhibits good scalability with an increasing number of Spark executors.
Conclusions:
- The developed Spark-based approach offers an efficient solution for parallelizing focal statistics on large raster datasets.
- This method effectively handles big terrain data, providing accurate and scalable spatial analysis.
- The findings contribute to advancing geo-raster spatial analysis in the context of big data.
Related Concept Videos
01:20Parallel Processing
442
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
442
01:25Methods of Obtaining Topography
181
Topography involves measuring and mapping land elevations, natural features, and artificial structures to create accurate representations of the terrain. Topographic surveying relies on traditional and modern methods, each with distinct advantages and limitations.Traditional Surveying Methods:Transit stadia surveys and plane table surveys were widely used traditional surveying methods. These techniques relied on instruments like theodolites and stadia rods for measuring distances and angles,...
181
01:24Area Computation by the Alternative Coordinate Method
340
The alternative coordinate method, also known as the Shoelace Formula, is a technique for determining the area of a traverse using Cartesian coordinates. This method relies on the sequential arrangement of x and y coordinates for each point of the shape, ensuring accuracy and ease of application.In this approach, each corner's x and y coordinates are listed as fractions, with the x-coordinate as the numerator and the y-coordinate as the denominator. These coordinates are arranged sequentially...
340
01:20Cluster Sampling Method
13.7K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
13.7K
01:29Plotting of Topographic Maps
284
Topographic maps represent the Earth's surface features using contour lines, which connect points of equal elevation to create a two-dimensional representation of three-dimensional terrain. Creating a topographic map requires a systematic approach.Begin by plotting a scaled grid and marking intersections corresponding to the survey's elevation data points. Assign elevation values at these intersections to build the base map. Next, determine contour levels using a consistent contour interval,...
284
01:21Distributed Loads: Problem Solving
917
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
917