Fractional designs: a simulation study of usefulness in the social sciences

Fractional designs can be extremely useful in social science research, especially when a large number of factors is involved. Reluctance for the use of fractional designs seems to be warranted for two reasons: (1) In the social sciences, the amount of measurement error is often large, which may decrease the power, and (2) higher order interactions are assumed to be nonsignificant, which is difficult to guarantee without sufficient research. This simulation study shows the effects of measurement error and assumption violations under various conditions. It is concluded that fractional designs handle measurement error gracefully and that they are as powerful as a full design when equal degrees of freedom are available. Significant interaction effects can cause serious problems, especially in situations with low or intermediate measurement error, and can lead to erroneous conclusions. Only when estimated confounded effects are clearly not significant, the chance of a wrong decision is reasonably small. Therefore, fractional designs are especially warranted for the exclusion of irrelevant factors. However, we note pitfalls in the use of Version 1.0 of the program Trail Run from SPSS, Inc., to implement the procedures.