Interactive Visual Assessment for Text-to-Image Generation Models.

Area of Science:

• Computer Graphics
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Visual generation models show progress but face real-world deployment challenges.
• Current assessment methods are limited by fixed coverage, evolving difficulty, and data leakage.
• Complex generation models require more effective evaluation strategies.

Purpose of the Study:

• To propose DyEval, a dynamic, interactive visual assessment framework for text-to-image systems.
• To address limitations of traditional isolated assessment frameworks.
• To improve the comprehensive evaluation of visual generation models.

Main Methods:

• DyEval utilizes a Large Language Model (LLM)-powered framework for collaborative human-AI evaluation.
• An intuitive interface allows interactive exploration and analysis of model behaviors.
• Adaptive generation of diverse textual inputs probes model capability boundaries based on user feedback.
• A contextual reflection module analyzes failure triggers and patterns using LLM reasoning.

Main Results:

• DyEval identified up to 2.56 times more generation failures compared to conventional methods.
• The framework uncovered complex and rare failure patterns, including pronoun and cultural context generation issues.
• Qualitative and quantitative experiments validated DyEval's effectiveness.

Conclusions:

• DyEval offers a novel approach to visual generation model assessment, enhancing reliability.
• The framework provides valuable insights for improving generative models.
• DyEval has broad implications for advancing visual generation systems across domains.