Brain enlargement and dental reduction were not linked in hominin evolution

The large brain and small postcanine teeth of modern humans are among our most distinctive features, and trends in their evolution are well studied within the hominin clade. Classic accounts hypothesize that larger brains and smaller teeth coevolved because behavioral changes associated with increased brain size allowed a subsequent dental reduction. However, recent studies have found mismatches between trends in brain enlargement and posterior tooth size reduction in some hominin species. We use a multiple-variance Brownian motion approach in association with evolutionary simulations to measure the tempo and mode of the evolution of endocranial and dental size and shape within the hominin clade. We show that hominin postcanine teeth have evolved at a relatively consistent neutral rate, whereas brain size evolved at comparatively more heterogeneous rates that cannot be explained by a neutral model, with rapid pulses in the branches leading to later Homo species. Brain reorganization shows evidence of elevated rates only much later in hominin evolution, suggesting that fast-evolving traits such as the acquisition of a globular shape may be the result of direct or indirect selection for functional or structural traits typical of modern humans.