Simplifying software compliance: AI technologies in drafting technical documentation for the AI Act

Area of Science:

• Computer Science
• Law and Technology

Background:

• The European AI Act mandates technical documentation for AI systems.
• Compliance is complex, requiring expertise in both legal and technical domains.
• Small and medium-sized enterprises face significant challenges and costs in meeting these requirements.

Purpose of the Study:

• To investigate the utility of contemporary AI technologies in assisting software developers with AI Act technical documentation.
• To evaluate the effectiveness of AI tools in identifying documentation gaps against AI Act provisions.
• To compare AI-generated compliance assessments with expert legal judgments.

Main Methods:

• Case studies using open-source, high-risk AI systems.
• Application of AI tools, including ChatGPT (versions 3.5 and 4) and DoXpert.
• Collaboration with legal experts for evaluation of AI-generated assessments.
• Statistical analysis using Rank Biserial Correlation.

Main Results:

• Partial alignment observed between AI tool assessments and expert opinions.
• Significant issues identified with ChatGPT's performance in compliance assessment.
• DoXpert showed a moderate, statistically significant correlation with expert judgments.
• AI tools show potential in identifying documentation gaps for AI Act compliance.

Conclusions:

• AI tools can assist in alleviating the burden of AI Act compliance documentation.
• Combining AI analysis with human expertise offers a promising approach to responsible AI development.
• Further development and validation of AI tools are needed for robust regulatory compliance.