SlideAudit: A Dataset and Taxonomy for Automated Evaluation of Presentation Slides

Area of Science:

• Computer Science
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Automated evaluation of graphic designs, particularly presentation slides, remains an unresolved challenge.
• Existing methods lack comprehensive frameworks for identifying specific slide design flaws.

Purpose of the Study:

• Introduce SlideAudit, a novel dataset for automated slide evaluation.
• Develop and validate a taxonomy of slide design flaws.
• Assess the capability of AI models in identifying and remediating slide design issues.

Main Methods:

• Compiled and synthesized 2400 slides from diverse sources, including intentionally flawed examples.
• Developed a detailed taxonomy of slide design flaws in collaboration with design experts.
• Annotated the dataset using trained crowdsourcing and evaluated various large language models (LLMs) with different prompting strategies.

Main Results:

• AI models demonstrated limited accuracy in identifying slide design flaws, with F1 scores between 0.331 and 0.655.
• Prompting techniques that incorporated the developed taxonomy yielded the highest performance.
• A remediation study indicated significant slide improvement in 82.0% of cases, with 87.8% of improvements being superior when using the taxonomy.

Conclusions:

• The SlideAudit dataset and taxonomy provide a valuable resource for advancing automated slide evaluation.
• Current AI models require structured guidance, such as taxonomies, to effectively identify design flaws.
• The developed taxonomy enhances AI's potential for both evaluating and improving slide design quality.