Effect of the threshold nature of traits on heritability estimates obtained by linear model

In this research we used stochastic method to simulate litter size data under assumption that the phenotype of the trait of interest is categorically distributed with a liability of normally distributed background effect. For the phenotypic categorical measurements, normal distribution of liability was assumed. Analysis was conducted by DFREML for phenotypic categorical data set. In the simulation process, we applied different true background heritabilities in combination with different threshold points. The points with the lowest heritability had the lowest variation in estimated heritability and those with the highest heritability had the highest variation. Results clearly show that by moving threshold points from the population mean towards the two tails of the distribution and also by increasing the true heritability, the differences between estimated heritability and true heritability increases. The linear model has always underestimated true heritabilities. It is possible to use our estimates to obtain a confidence range for true heritability when we have information about approximate location of threshold and estimates of heritability obtained by linear model.