A Critical Review of Network-Based and Distributional Approaches to Semantic Memory Structure and Processes

Some of the earliest work on understanding how concepts are organized in memory used a network-based approach, where words or concepts are represented as nodes, and relationships between words are represented by links between nodes. Over the past two decades, advances in network science and graph theoretical methods have led to the development of computational semantic networks. This review provides a modern perspective on how computational semantic networks have proven to be useful tools to investigate the structure of semantic memory as well as search and retrieval processes within semantic memory, to ultimately model performance in a wide variety of cognitive tasks. Regarding representation, the review focuses on the distinctions and similarities between network-based (based on behavioral norms) approaches and more recent distributional (based on natural language corpora) semantic models, and the potential overlap between the two approaches. Capturing the type of relation between concepts appears to be particularly important in this modeling endeavor. Regarding processes, the review focuses on random walk models and the degree to which retrieval processes demand attention in pursuit of given task goals, which dovetails with the type of relation retrieved during tasks. Ultimately, this review provides a critical assessment of how the network perspective can be reconciled with distributional and machine-learning-based perspectives to meaning representation, and describes how cognitive network science provides a useful conceptual toolkit to probe both the structure and retrieval processes within semantic memory.