Enhancing Ontology Engineering with Large Language Models: A Stage-Wise Human-in-the-Loop Study

Ontology engineering plays a critical role in modelling structured knowledge and ensuring semantic interoperability in digital healthcare. However, manually developing ontologies is time-consuming and dependent on human expertise. This study investigates the utility of large language models (LLMs) on enhancing ontology development with a human-in-the-loop workflow. LLMs were integrated at three checkpoints: (1) term extraction, (2) class and hierarchy definition, and (3) relation specification. Two experimental settings were tested, including LLM-only generation and LLM with additional manual inputs. Results were compared against the stage-wise outputs from a manually constructed ontology using precision, recall, and F1 metrics. Findings showed that human-in-the-loop input consistently improves output quality, while LLMs perform better in early conceptualisation stages than later formalisation stages. This study highlights the complementary strengths of transformer-based reasoning and human validation and presents a reproducible framework for stage-wise evaluation of LLM-assisted ontology engineering. The findings offer practical guidance for integrating generative AI into ontology-driven health data modelling and standardisation.