Two-stage method to remove population- and individual-level outliers from longitudinal data in a primary care...

Area of Science:

• Data Science
• Biostatistics
• Epidemiology

Background:

• Primary care databases contain longitudinal health data with potential for extreme or erroneous values.
• Distinguishing true extreme values from recording errors is crucial for accurate health research.

Purpose of the Study:

• To evaluate methods for identifying outliers in longitudinal health records.
• To develop and validate a robust approach for outlier detection in large population datasets.

Main Methods:

• A two-stage outlier identification strategy was employed.
• Population-level outliers were identified using Health Survey for England data.
• Individual-level outliers were detected using a random-effects model with patient-level standardized residuals.

Main Results:

• The proposed two-stage method identified 1550 population outliers and 75 individual outliers.
• This approach proved more efficient in identifying true outliers compared to existing methods.
• The method successfully identified outliers at both population and individual levels.

Conclusions:

• A new, effective two-stage approach for outlier identification in longitudinal data is proposed.
• The method enhances the accuracy of health data by differentiating true extreme values from recording errors.
• This technique improves data quality for epidemiological and clinical research.