Autodidactic dense anatomical models

Humans effortlessly interpret images by parsing them into part-whole hierarchies. Yet, deep learning models, despite excelling at capturing multi-level features, often fail to explicitly encode these part-whole hierarchies-an essential aspect of medical imaging, which boasts anatomical hierarchies in nature. To address this limitation, we introduce Adam-v2, a self-supervised learning framework that explicitly learns to encode inherent part-whole hierarchies within medical images through three key branches: (1) "localizability", which acquires discriminative representations to distinguish different anatomical structures; (2) "composability", which learns each anatomical structure in a parts-to-whole manner; and (3) "decomposability", which comprehends each anatomical structure in a whole-to-parts manner. Our extensive experiments showcase Adam-v2's advanced capability in anatomy understanding, unveiled through its embeddings (Eve-v2). Particularly, Eve-v2 demonstrates a zero-shot understanding of anatomy as revealed through its learned properties: ➀ preserving localizability of anatomical structures and ➁ encoding part-whole relations of anatomical structures as well as its emergent properties: ➂ understanding anatomical layouts via interpolation and extrapolation, ➃ associating each image pixel with (dense) semantic embeddings, ➄ recognizing anatomical symmetries, ➅ generating embeddings consistent across scales, and ➆ matching anatomical structures across images of the same patient with different diseases, images of different patients, and augmented views of the same image. Adam-v2 also offers robust and generalizable representations and stands out in ➇ few-shot learning, ➈ full-transfer learning, and ➉ novelty and anomaly detection. These capabilities and performance directly stem from our crafted anatomy learning strategy, which explicitly constructs hierarchies for distinct anatomical structures from unlabeled medical images. Project page: GitHub.com/JLiangLab/Eden.