Meta-representations as representations of processes

In this study, we explore how the notion of meta-representations in higher-order theories (HOT) of consciousness can be implemented in computational models. HOT suggests that consciousness emerges from meta-representations, which are representations of first-order sensory representations. However, translating this abstract concept into a concrete computational model, such as those used in artificial intelligence, presents a theoretical challenge. For example, a simplistic interpretation of meta-representation as a representation of representation makes the notion rather trivial and ubiquitous. Here, as a foundational step toward understanding meta-representations, we propose a refined computational interpretation that focuses specifically on process-level representations. Contrary to the simplistic view of meta-representations as mere transformations of the first-order representational states or confidence estimates, we argue that meta-representations represent the computational processes that generate first-order representations, building on the Radical Plasticity Thesis by Cleeremans (2011). https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2011.00086.) This presents a process-oriented view whereby meta-representations capture the qualitative aspect of how sensory information is transformed into first-order representations. As a proof-of-concept of this formulated notion of meta-representation, we constructed "meta-networks" designed to explicitly model meta-representations within deep learning architectures while methodologically isolating process representations from specific sensory activations to avoid confounding effects. Specifically, we constructed meta-networks by implementing autoencoders of first-order neural networks. In this architecture, the latent spaces embedding those first-order networks correspond to the meta-representations of first-order networks. By applying meta-networks to embed neural networks trained to encode visual and auditory datasets, we show that the meta-representations of first-order networks successfully capture the qualitative aspects of those networks by separating the visual and auditory networks in the meta-representation space. We argue that such meta-representations would be useful for quantitatively comparing and contrasting the qualitative differences of computational processes. While whether such meta-representational systems exist in the human brain remains an open question, this formulation of meta-representation offers a new empirically testable hypothesis that there are brain regions that represent the processes of transforming a representation in one brain region to a representation in another brain region. Furthermore, this form of meta-representations might underlie our ability to describe the qualitative aspect of sensory experience or qualia.