Robustness of Deep-Learning-Based RF UAV Detectors | JoVE Visualize

Area of Science:

Electrical Engineering
Computer Science
Defense Technology

Background:

The increasing prevalence of small, low-cost unmanned aerial vehicles (UAVs) presents a significant security challenge.
Passive radio frequency (RF) spectrum scanning, enhanced by machine learning (ML) and deep learning (DL), is a key method for detecting UAV control signals.
Existing research focuses on optimizing ML/DL model accuracy, often overlooking robustness across diverse operational scenarios.

Purpose of the Study:

To address the critical gap in evaluating the robustness of UAV detection systems.
To introduce a novel dataset specifically designed to assess detector performance under varying channel conditions.
To compare the robustness of different ML/DL architectures, specifically Convolutional Neural Networks (CNNs) and coefficient classifiers, in challenging RF environments.

Main Methods:

Development of a new dataset featuring multiple test data categories based on diverse channel conditions, distinct from the training data pool.
Evaluation of existing UAV detection models, including CNNs and coefficient classifiers, using the newly created dataset.
Quantitative analysis of classifier performance, particularly under low signal-to-noise ratio (SNR) conditions and varied RF channel characteristics.

Main Results:

Image classifiers, specifically CNNs, demonstrated approximately 40% greater robustness compared to coefficient classifiers under low SNR conditions.
CNN classifiers maintained consistent accuracy across various RF channel conditions not encountered during training.
Coefficient classifiers exhibited significant performance degradation, ranging from partial to complete failure, depending on specific channel conditions.

Conclusions:

Robustness is a critical, yet often overlooked, performance metric for UAV detection systems.
CNN-based approaches offer superior robustness for detecting UAVs via RF signal analysis, especially in adverse channel conditions.
The developed dataset provides a valuable resource for future research aimed at enhancing the reliability of counter-UAV technologies.