Effects of selective serotonin antagonism on central neurotransmission

Aggression and cannibalism in laying hens can differ in intensity and degree due to many factors, including genetics. Previous behavioral analysis of 2 strains of White Leghorns, DeKalb XL (DXL) and HGPS (a group-selected line for high group productivity and survivability), revealed high and low aggressive phenotypes, respectively. However, the exact genetic mechanisms mediating aggressiveness are currently unknown. Analysis of serotonin (5-HT) mediation of aggression in subordinate hens of these strains revealed increases in aggression in DXL hens following antagonism of the 5-HT1A receptor and in HGPS hens following antagonism of the 5-HT1B receptor. Here, we investigate the different neurotransmitter response in the hypothalamus and raphe nucleus mediating these aggressive responses to receptor antagonism. Elevated aggressive response to 5-HT1B antagonism by HGPS hens was also accompanied by a decrease in raphe nucleus dopamine (DA) and an increase in DA turnover. Increased aggressiveness in DXL hens did not coincide with a reduction in raphe nucleus 5-HT or turnover (as indicated by 5-hydroxyindoleacetic acid levels) following 5-HT1A antagonism. A reduction in 5-hydroxyindoleacetic acid (but not 5-HT) was seen in HGPS hens treated with 5-HT1A antagonist; however, these hens exhibited no change in aggressive behaviors. Our data show evidence of different heritable mechanisms of neurotransmitter regulation of aggressive response, specifically heritable differences in the interaction between 5-HT and catecholamines in regulating aggression.