Expanding amelogenin peptide panels to address target-specific loss in enamel-based sex determination

Proteomics applied in modern archaeological studies allows for the detection of relevant molecular information with remarkable sensitivity from minute sample amounts, providing reliable information when morphology or DNA-based methods fail. One such application is sex determination using proteomics analysis of tooth enamel. Amelogenin is one of the most abundant proteins of the tooth enamel. In many mammals, including humans, the corresponding gene is located on the sex chromosomes, thus forming X- and Y-isoforms (the latter being specific to males). Current targeted proteomics analysis methods use peptides from the tyrosine-rich amelogenin peptide (TRAP) region, which contains isoform-specific amino acid motifs. In this study, we investigated 100 teeth, 91 from subadults and nine from poorly preserved adult skeletons, excavated from a late medieval burial site in Virklund, Denmark. The initial analysis using the established TRAP-only targeted proteomics method revealed that 11 samples showed reduced or absent detection of AMELX peptides (five with complete absence and six with only a single peptide detected), resulting in low-confidence or inconclusive classifications. Using discovery proteomics, we expanded the peptide panel with two peptides specific to the AMELX C-terminal domain. Incorporating these peptides restored confident sex determination in all previously inconclusive cases, without compromising analytical performance or sensitivity. This suggests the C-terminal region of amelogenin as a stable and diagnostically valuable source of peptide markers. More broadly, the success of the expanded panel highlights the synergistic value of discovery and targeted proteomics to accommodate sample-dependent variability in archaeological enamel samples.