Related Experiment Video
Updated: Apr 7, 2026
09:43
Databases to Efficiently Manage Medium Sized, Low Velocity, Multidimensional Data in Tissue Engineering
Published on: November 22, 2019
6.9K
An Efficient Approach for Web Indexing of Big Data through Hyperlinks in Web Crawling
R Suganya Devi1, D Manjula1, R K Siddharth1
1Department of Computer Science and Engineering, College of Engineering, Guindy, Anna University, Chennai 600025, India.
Thescientificworldjournal
|July 3, 2015
Summary
This study introduces an improved web crawling method that extracts not just links but also associated metadata. This approach streamlines data analysis for big data applications by reducing pre-processing workload.
Area of Science:
- Computer Science
- Information Retrieval
Background:
- The World Wide Web's growth presents challenges for search engines and data analysis.
- Current web crawling methods often focus only on link extraction, increasing post-processing burdens.
Purpose of the Study:
- To develop a web crawling technique that retrieves both links and associated metadata.
- To facilitate easier processing of big data obtained from web crawling.
Main Methods:
- A modified Depth First Search algorithm is employed for hierarchical link crawling.
- Web page source code is accessed to extract metadata like titles, keywords, and descriptions.
Main Results:
- Successfully crawled links from specified URLs.
- Extracted essential metadata (title, keywords, description) associated with crawled links.
- Demonstrated a method to reduce the analytical workload for big data.
Conclusions:
- The proposed web crawling method enhances data processing efficiency.
- Extracting metadata alongside links provides richer data for analysis.
- This approach is crucial for effective big data analysis in the context of web crawling.
Related Concept Videos
00:56Extraction: Advanced Methods
1.3K
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
1.3K
01:21Heuristics
898
Heuristics are problem-solving strategies that use mental shortcuts to simplify decision-making. Unlike algorithms, which must be followed precisely to achieve a correct result, heuristics offer a general problem-solving framework. They save time and energy but can sometimes lead to less rational decisions.
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...
898