Tina: A diffusion neural network for generating personalized AI models from text prompts.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Generative artificial intelligence (GenAI) has shown significant advancements in various modalities like text, image, and video generation.
• Existing GenAI models primarily focus on content creation rather than functional model generation.

Purpose of the Study:

• To investigate the potential of GenAI for text-to-model generation, specifically mapping semantic task descriptions to functional neural network parameters.
• To introduce Tina, a novel text-conditioned neural network diffusion model for personalized classification.

Main Methods:

• Developed Tina, a diffusion transformer model conditioned on contrastive language-image pre-training (CLIP)-embedded task descriptions.
• Utilized text prompts at inference time to generate personalized classifiers.

Main Results:

• Tina successfully generated high-quality personalized classifiers for both natural and medical images.
• Demonstrated in-distribution and out-of-distribution personalization capabilities.
• Showcased support for zero-shot/few-shot image prompts, generalization to unseen classes, and scalability to complex tasks.

Conclusions:

• Established text-to-model GenAI as a viable paradigm for on-demand personalization.
• Highlighted Tina's effectiveness in creating personalized classifiers from natural language instructions.
• Proposed a new avenue for human-AI interaction through text-based model generation.