Phylogenetic reduction of the magnocellular red nucleus in primates and inter-subject variability in humans

The red nucleus is part of the motor system controlling limb movements. While this seems to be a function common in many vertebrates, its organization and circuitry have undergone massive changes during evolution. In primates, it is sub-divided into the magnocellular and parvocellular parts that give rise to rubrospinal and rubro-olivary connection, respectively. These two subdivisions are subject to striking variation within the primates and the size of the magnocellular part is markedly reduced in bipedal primates including humans. The parvocellular part is part of the olivo-cerebellar circuitry that is prominent in humans. Despite the well-described differences between species in the literature, systematic comparative studies of the red nucleus remain rare.

We therefore mapped the red nucleus in cytoarchitectonic sections of 20 primate species belonging to 5 primate groups including prosimians, new world monkeys, old world monkeys, non-human apes and humans. We used Ornstein-Uhlenbeck modelling, ancestral state estimation and phylogenetic analysis of covariance to scrutinize the phylogenetic relations of the red nucleus volume.

We created openly available high-resolution cytoarchitectonic delineations of the human red nucleus in the microscopic BigBrain model and human probabilistic maps that capture inter-subject variations in quantitative terms. Further, we compared the volume of the nucleus across primates and showed that the parvocellular subdivision scaled proportionally to the brain volume across the groups while the magnocellular part deviated significantly from the scaling in humans and non-human apes. These two groups showed the lowest size of the magnocellular red nucleus relative to the whole brain volume and the largest relative difference between the parvocellular and magnocellular subdivision.

That is, the red nucleus has transformed from a magnocellular-dominated to a parvocellular-dominated station. It is reasonable to assume that these changes are intertwined with evolutionary developments in other brain regions, in particular the motor system. We speculate that the interspecies variations might partly reflect the differences in hand dexterity but also the tentative involvement of the red nucleus in sensory and cognitive functions.