Benchmarking state of the art website embedding methods for effective processing and analysis in the public sector

Area of Science:

• Computer Science
• Information Retrieval
• Web Science

Background:

• Machine understanding of websites is essential for various applications.
• Website embedding is critical for tasks like monitoring local government digital transformation.
• Evaluating different website embedding methods is necessary to find optimal solutions.

Purpose of the Study:

• To compare state-of-the-art website embedding methods for machine understanding.
• To assess the effectiveness of visual, mixed, and textual embedding models for local government website monitoring.
• To evaluate model performance using zero-shot and transfer learning approaches.

Main Methods:

• Comparison of visual, mixed, and textual website embedding models.
• Baseline model: embedding website header section.
• Performance evaluation using zero-shot and transfer learning on three datasets.
• Metrics: embedding scoring, classification performance (precision, F1-score), and processing time.

Main Results:

• Homepage2Vec (visual and textual combination) performed best in zero-shot learning across datasets.
• TF-IDF & FNN (text-based) achieved superior performance in transfer learning (clustering, precision, F1-score).
• The baseline model offers a faster alternative, 1.88x speed improvement with a minor 10% F1-score decrease.

Conclusions:

• Website embedding methods vary in effectiveness depending on the learning approach (zero-shot vs. transfer learning).
• TF-IDF & FNN is optimal for transfer learning tasks requiring high accuracy.
• The baseline model provides a practical, time-efficient solution for large-scale data processing.