Optimal Nonparametric Inference with Two-Scale Distributional Nearest Neighbors

Area of Science:

• Statistics
• Nonparametric Statistics
• Machine Learning

Background:

• Weighted nearest neighbors (WNN) is a flexible nonparametric tool for mean regression.
• Distributional nearest neighbors (DNN) uses bagging to create WNN estimators.
• Existing DNN methods lack distributional results and optimal convergence rates for smooth functions.

Purpose of the Study:

• To address the limitations of DNN estimators, particularly bias issues in higher-order smoothness cases.
• To develop a bias-reduced DNN estimator that achieves optimal nonparametric convergence rates.
• To establish theoretical properties and practical implementation tools for the proposed estimator.

Main Methods:

• Introduced a bias reduction approach by combining two DNN estimators with different subsampling scales, creating the two-scale DNN (TDNN) estimator.
• Provided an equivalent representation of TDNN as a WNN estimator with explicit, potentially negative, weights.
• Established asymptotic normality for both DNN and TDNN estimators.

Main Results:

• The TDNN estimator achieves the optimal nonparametric rate of convergence under fourth-order smoothness conditions, overcoming DNN's bias limitations.
• Theoretical analysis confirmed asymptotic normality for DNN and TDNN estimators.
• Developed variance and distribution estimators for TDNN using jackknife and bootstrap techniques for practical inference.

Conclusions:

• The TDNN estimator offers a significant improvement over standard DNN, achieving optimal convergence rates and enabling valid statistical inference.
• The theoretical results and practical implementation tools (variance/distribution estimators) support the use of TDNN for nonparametric regression.
• The study demonstrates the effectiveness of TDNN through simulations and a real data application.