A Calibrated Multi-Dimensional Evaluation Framework for Diffusion-Based Radio Frequency Signal Generation

Area of Science:

• Electrical Engineering
• Machine Learning
• Signal Processing

Background:

• Current evaluation of Radio Frequency (RF) signal generative models lacks standardized, calibrated metrics.
• Existing methods often import unverified metrics from computer vision, hindering reliable model comparison.

Purpose of the Study:

• To introduce a novel, multi-dimensional evaluation framework for RF signal generative models.
• To establish reproducible standards for assessing and comparing RF generative models.

Main Methods:

• Developed a framework with ten metrics across six layers, incorporating distribution-level aggregation and real-real baseline normalization.
• Utilized signal-to-noise ratio (SNR)-modulation stratification and multi-dimensional coverage principles.
• Established real-real baselines by comparing independent subsets of authentic signals to set performance ceilings.

Main Results:

• Identified that linear Short-Time Fourier Transform preprocessing causes generation failure in analog amplitude modulation, which logarithmic compression resolves.
• Discovered that noise-prediction objectives suppress gradients for amplitude-modulated signals; signal-prediction improves temporal fidelity.
• Demonstrated significant improvements in RF signal generation quality through framework application and proposed adjustments.

Conclusions:

• The proposed framework provides a systematic and calibrated approach for evaluating RF generative models.
• The findings highlight critical preprocessing and training objective choices for improving analog and digital RF signal generation.
• Established reproducible standards for comparative assessment in RF signal modeling.