Navigating the Seas of AI: Effectiveness of Small Language Models on Edge Devices for Maritime Applications

This paper explores the feasibility of employing small language models (SLMs) on edge devices powered by batteries in environments with limited/no internet connectivity. SLMs in fact offer significant advantages in such scenarios due to their lower resource requirements with respect to large language models. The use case in this study is maritime navigation-in particular, the documentation on Sailing Directions (Enroutd) of the World Port Index (WPI) provided by the National Geospatial-Intelligence Agency (NGA), which provides information that cannot be shown graphically on nautical charts and is not readily available elsewhere. In this environment, response immediacy is not critical, as users have sufficient time to query information while navigating and planning activities, making edge devices ideal for running these models. On the contrary, the response quality is fundamental. For this reason, given the constrained knowledge of SLMs in maritime contexts, we investigate the use of the retrieval-augmented generation (RAG) methodology, integrating external information from sailing directions. A comparative analysis is presented to evaluate the performance of various state-of-the-art SLMs, focusing on response quality, the effectiveness of the RAG component, and inference times.