Sample size requirements for the design of reliability studies: precision consideration

Area of Science:

• Statistics
• Multilevel Modeling
• Psychometrics

Background:

• The intraclass correlation coefficient (ICC) from one-way random-effects models quantifies group member resemblance.
• Reporting confidence intervals (CIs) for ICC is standard practice in reliability studies.
• Existing sample size methods often rely on large-sample approximations, potentially limiting precision.

Purpose of the Study:

• To develop exact sample size determination procedures for precise interval estimation of the ICC.
• To address various allocation and cost structures relevant to reliability study design.
• To improve upon the accuracy of sample size calculations compared to existing methods.

Main Methods:

• Development of exact sample size procedures for ICC confidence interval estimation.
• Utilized iterative algorithms for computations, as explicit formulas are not available.
• Evaluated accuracy based on expected interval width and assurance probability criteria.

Main Results:

• The proposed exact methods provide more accurate sample size recommendations than large-sample approximations.
• These procedures are applicable under diverse allocation and cost constraints.
• The study highlights deficiencies in current sample size methodologies for ICC reliability studies.

Conclusions:

• Exact sample size procedures enhance the precision of confidence interval estimation for the ICC.
• These methods offer a more rigorous approach to designing reliability studies.
• The expanded methodology fills a gap in the existing literature for sample size determination in ICC research.