Addressing the Problem of Hard-to-Reach Unpublished Data from Theses in University Repositories.

Area of Science:

• Environmental Science
• Hydrology
• Data Science
• Information Retrieval

Background:

• University theses contain valuable, often unpublished, environmental and hydrological data.
• PDF format and inadequate repository search tools hinder data access and analysis.
• Manual data extraction from theses is time-consuming and inefficient.

Purpose of the Study:

• To develop an automated solution for extracting groundwater data from university theses.
• To overcome the limitations of keyword-based searches in accessing gray literature.
• To create a searchable groundwater database for the Arequipa region, Peru.

Main Methods:

• Optical Character Recognition (OCR) for text extraction from PDFs.
• Python scripts for initial keyword scoring.
• Large Language Models (LLMs), including Google Gemini and Ollama, for semantic content analysis.
• Data extraction and organization into formats like Excel spreadsheets.

Main Results:

• Successful identification and extraction of key groundwater data (e.g., water quality, well locations).
• High accuracy confirmed through subsequent manual checks.
• Development of a system enabling swift, contextual querying of numerous documents.

Conclusions:

• The developed methodology effectively addresses challenges in accessing and utilizing data from gray literature.
• The system enhances the accessibility and management of regional groundwater data.
• The approach is scalable and adaptable for broader applications in scientific discovery across disciplines.