A dataset of neural network architectures generated via large language models

Area of Science:

• Computer Science
• Artificial Intelligence
• Software Engineering

Background:

• Neural networks (NNs) are critical for decision-making, necessitating robust verification tools.
• Existing tools for NN code verification, refactoring, and migration aim to ensure correctness and maintainability.
• Evaluating these tools is hindered by a lack of diverse, publicly available NN datasets.

Purpose of the Study:

• To address the dataset scarcity for evaluating NN verification tools.
• To generate a comprehensive and diverse dataset of neural networks.
• To create a benchmark for assessing the effectiveness of NN reliability and adaptability tools.

Main Methods:

• Leveraging large language models (LLMs) for automated generation of neural network samples.
• Designing the dataset to encompass diverse layer types, input data types, and tasks.
• Employing static analysis and symbolic tracing to validate the correctness of generated networks.
• Generating a total of 608 unique neural network samples.

Main Results:

• Successfully generated a dataset of 608 diverse neural networks.
• Validated the correctness and consistency of the generated networks.
• The dataset covers various layer types and accommodates multiple input data formats and tasks.

Conclusions:

• The generated dataset provides a valuable benchmark for evaluating neural network verification tools.
• Public availability of this dataset will advance research in neural network reliability and adaptability.
• Facilitates systematic evaluation of tools for NN code verification, refactoring, and migration.