Are children's rule-assessment classifications invariant across instances of problem types?

Wilkening and Anderson recently criticized Siegler's rule-assessment methodology as being insufficiently diagnostic of some rules that may be used to solve tasks like the balance scale. An implication of this criticism is that the rule-assessment method may yield classifications that vary with the problems used to assess knowledge. We tested this implication by varying the size of the product difference within problem types for both the balance-scale and the inclined-plane tasks. Many children's classifications differed from problem set to problem set. Children tended to achieve more developmentally sophisticated classifications with larger product differences, and variability in classifications was attributable to greater accuracy on all problem types with larger product differences. The variability in accuracy on the simple problem types is predicted by neither Siegler's nor Wilkening and Anderson's analyses. Possible explanations for these findings were discussed.