Structural adaptation of vertebrate endonuclease G for 5-hydroxymethylcytosine recognition and function

Modified DNA bases functionally distinguish the taxonomic forms of life-5-methylcytosine separates prokaryotes from eukaryotes and 5-hydroxymethylcytosine (5hmC) invertebrates from vertebrates. We demonstrate here that mouse endonuclease G (mEndoG) shows specificity for both 5hmC and Holliday junctions. The enzyme has higher affinity (>50-fold) for junctions over duplex DNAs. A 5hmC-modification shifts the position of the cut site and increases the rate of DNA cleavage in modified versus unmodified junctions. The crystal structure of mEndoG shows that a cysteine (Cys69) is positioned to recognize 5hmC through a thiol-hydroxyl hydrogen bond. Although this Cys is conserved from worms to mammals, a two amino acid deletion in the vertebrate relative to the invertebrate sequence unwinds an α-helix, placing the thiol of Cys69 into the mEndoG active site. Mutations of Cys69 with alanine or serine show 5hmC-specificity that mirrors the hydrogen bonding potential of the side chain (C-H < S-H < O-H). A second orthogonal DNA binding site identified in the mEndoG structure accommodates a second arm of a junction. Thus, the specificity of mEndoG for 5hmC and junctions derives from structural adaptations that distinguish the vertebrate from the invertebrate enzyme, thereby thereby supporting a role for 5hmC in recombination processes.