Recognition Mechanism of Serotonin by a G-Quadruplex-Duplex Hybrid Aptamer

Serotonin is a key neurotransmitter, and aptamer-based tools using the 44-nt Apt44 have been successfully developed for its in vitro and in vivo detection. Nevertheless, the structural basis of recognition by this aptamer remains unclear. Here, we report high-resolution NMR structures of Apt38, a 6-nt truncated variant in the third loop of Apt44, in free and serotonin-bound states. Both structures reveal a two-layered antiparallel chair-type G-quadruplex core with three edgewise loops and a terminal duplex, forming a G-quadruplex-duplex hybrid structure. Serotonin binds at the G-quadruplex-duplex junction by stacking directly onto the G-tetrad plane without contacting the duplex, representing a newly identified binding mode at this hotspot junction. This binding is stabilized by stacking, electrostatic attraction, hydrogen bonding, and hydrophobic contacts. Apt38 is preorganized for binding, whereas the longer third loop of Apt44 introduces conformational dynamics into the G-quadruplex scaffold, which enables a pronounced binding-triggered conformational switch in PBS buffer, explaining its sensing mechanism. Our work reveals the recognition and sensing mechanism of the serotonin aptamer and provides a framework for aptamer design in serotonin biosensing.