Synthetic aircraft RS image modelling based on improved conditional GAN joint embedding network | JoVE Visualize

Area of Science:

Remote Sensing Technology
Computer Vision
Artificial Intelligence

Background:

Developing efficient remote sensing (RS) models for diverse aircraft types is challenging due to feature entanglement in fine-class modeling.
Generative models offer a promising approach for simulating RS data, but existing methods struggle with subclass distinctions.

Purpose of the Study:

To introduce a novel, first-generation realistic aircraft type simulation system (ATSS-1) for high-quality RS image modeling.
To address the challenge of feature entanglement in multi-class aircraft modeling within RS.
To enhance the development of automatic and realistic RS simulation technologies.

Main Methods:

Developed ATSS-1, incorporating an adaptive weighted conditional attention generative adversarial network and a joint geospatial embedding (GE) network.
Utilized an adaptive weighted conditional batch normalization attention block to resolve subclass entanglement by reassigning intra-class characteristic responses.
Implemented an asymmetric residual self-attention module to capture finer spatial representations by establishing remote region asymmetric relationships.
Employed a GE network with a selected prior distribution (z) to explore the mapping between RS scenes and generated sample spaces.

Main Results:

ATSS-1 successfully achieved fine modeling of seven aircraft types within real RS scenes.
The adaptive attention mechanism effectively mitigated feature entanglement issues between different aircraft subclasses.
The GE network facilitated a clear mapping between input RS data and the latent space of generated samples.
Model performance was validated using the OPT-Aircraft_V1.0 dataset, alongside MNIST and Fashion-MNIST for broader testing.

Conclusions:

ATSS-1 demonstrates significant effectiveness in realistic aircraft type simulation for remote sensing.
The developed system advances the capability for automatic and high-fidelity RS simulation.
This work paves the way for further innovations in generative modeling for specialized RS applications.