Population substructure affects kinship testing in multi-ethnic areas of China

The likelihood ratio (LR) is a recommended metric for assessing the strength of genetic information in relationship testing, one of the most important tasks in forensic science. LR calculation incorporate population frequencies, which is affected by population substructure. This study utilized population frequency data from 18 short tandem repeat (STR) loci across 13 Chinese populations, encompassing both majority and minority ethnic groups. Six kinship types were constructed for each population. To understand the impact of population substructure on kinship testing, LRs were calculated using various frequency data: population-specific allele frequencies, national allele frequencies, and national allele frequencies adjusted with overall national F_ST or population-specific F_ST. LRs were also compared using the cutoff and comparison methods. The study found that LRs calculated using national allele frequencies tend to be the largest, which could overestimate the degree of relatedness compared to population-specific allele frequencies. Fst correction decreased the LR values, resulting in more conservative outcomes and suggested more distant relationships. While the F_ST correction had a minimal effect on the majority and some minority populations across different kinships, it was insufficiently conservative for more isolated minority populations when the overall national F_ST was applied. In conclusion, for isolated subpopulations with F_ST values above the national average, utilizing population-specific allele frequencies and applying higher F_ST values (e.g. 0.03 or 0.05) leads to more accurate and conservative inferences of relatedness. In contrast, for other groups, national frequencies without F_ST correction appear sufficient for relationship testing.