A case study of forensic psychiatry experts' reports analysis through large language models

Area of Science:

• Forensic Psychiatry
• Artificial Intelligence
• Clinical Informatics

Background:

• Forensic psychiatry increasingly utilizes artificial intelligence (AI) for enhanced decision-making and outcome prediction.
• Extracting comprehensive data from complex forensic psychiatric reports is time-consuming and challenging.

Purpose of the Study:

• To assess the feasibility and performance of a large language model (LLM), specifically GPT-4o, in extracting clinical and non-clinical variables from forensic psychiatric reports.
• To explore the potential of AI in semi-automating data collection for forensic psychiatric research.

Main Methods:

• Utilized GPT-4o to process two authentic forensic psychiatric expert reports.
• Employed custom queries to extract relevant clinical and non-clinical data related to criminal responsibility and social dangerousness.
• Evaluated the model's capability to identify key information and generate summarized outputs.

Main Results:

• The LLM successfully extracted information from the forensic psychiatric reports.
• The system demonstrated the ability to generate a summarized version of the extracted data.
• Challenges remain in identifying the most critical information for meaningful synthesis in this specialized domain.

Conclusions:

• AI, particularly LLMs, holds significant potential for semi-automated or automated data extraction from forensic psychiatric reports.
• This approach can facilitate the creation of large datasets for future research and analysis in forensic psychiatry.
• Further development is needed to refine AI's ability to synthesize complex, specialized information effectively.