Information temperature and macroscopic descriptors of additive Nth-order Markov chains

Large-scale language models (LLMs) operate in extremely high-dimensional state spaces, where both token embeddings and their hidden representations create complex dependences that are not easily reduced to classical Markov structures. In this paper we explore a theoretically solvable approximation of LLM dynamics using Nth-order additive Markov chains. Such models allow the conditional probability of the next token to be decomposed into a superposition of contributions from multiple historical depths, reducing the combinatorial explosion typically associated with high-order Markov processes. The main result of the work is the establishment of a correspondence between an additive multistep chain and a chain with a stepwise memory function. This correspondence allows the introduction of the concept of information temperature not only for stepwise but also for additive Nth-order Markov chains.