A topic-based framework for interpretable and sparse semantic representations

In computational approaches to language, semantic representations produced by modern deep learning systems are often encoded as high-dimensional latent vectors that are difficult to interpret. By contrast, cognitive science has emphasized the need for semantic representations with interpretable and cognitively meaningful structures. Although researchers have explored concept representations using corpus-driven distributed embeddings and various feature-based paradigms, a scalable and interpretable framework that can represent linguistic materials across different levels of granularity remains an open methodological challenge. Recent advances in pretrained language models have led to topic modeling methods that leverage these models, such as BERTopic, which uses sentence-level embeddings to derive clusters corresponding to human-interpretable topics. Building on this line of work, the present study introduces a framework, SISTR, for representing Semantics as Interpretable and Sparse Topic-based Representations. The framework extracts interpretable topics from the corpus and models the semantics of linguistic materials via their relations to the derived topics. This framework is designed to accommodate linguistic materials across different levels and languages. We evaluated the interpretability of the resulting topic dimensions through behavioral experiments. We applied this framework to word association tasks and semantic judgments, comparing model predictions with human behavioral data. The results suggest that SISTR achieves competitive performance to existing semantic representation methods while providing sparse and interpretable semantic representations. Overall, SISTR provides a reusable semantic coordinate system that offers an interpretable bridge between linguistic data and cognitive theories of semantic representation.