Related Experiment Video
Updated: Jun 15, 2026
13:44
Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns
Published on: August 30, 2013
43.5K
BI-RADS-compliant structured mammography reporting using locally deployed large language models under privacy
Wenjun Sheng1,2, Yudong Wang2, Lianxiang Xiao3
1School of Physics and Electronic Engineering, Linyi University, Linyi, China.
European Radiology
|November 19, 2025
Summary
This study introduces a privacy-preserving method for structuring mammography reports using a fine-tuned, open-source large language model (LLM). The developed approach achieves high accuracy and structural integrity, offering a practical, locally deployable solution for medical institutions.
Area of Science:
- Artificial Intelligence in Medical Imaging
- Natural Language Processing for Healthcare Data
- Medical Informatics and Data Structuring
Background:
- Free-text mammography reports present challenges in standardization and structured data extraction.
- Existing solutions for structuring reports may compromise data privacy, adaptability, or incur high costs.
- There is a need for privacy-preserving, adaptable, and cost-effective methods for mammography report analysis.
Purpose of the Study:
- To develop and evaluate a privacy-preserving method for structuring unstructured free-text mammography reports.
- To fine-tune an open-source large language model (LLM) for accurate and reliable data extraction.
- To assess the performance and structural integrity of the developed LLM-based system.
Main Methods:
- A multicenter study involving 7161 unstructured mammography reports.
- Supervised fine-tuning of the open-source Llama-3 model using pseudo-labels from a commercial model (Qwen-Max).
- Evaluation of 23 features using Precision, Recall, and F1-score; assessment of structural integrity via JSON format accuracy (JFA) and field integrity accuracy (FIA).
Main Results:
- The fine-tuned Llama-3 model achieved high performance (F1-score: 0.932) and complete structural integrity (FIA: 1.000).
- The model demonstrated significant gains over the base model in accuracy and structural integrity.
- While slightly below the commercial model, the fine-tuned model showed statistically significant performance, especially in specific categories.
Conclusions:
- The developed method effectively structures mammography reports using a locally fine-tuned, open-source LLM, enhancing privacy and local deployability.
- The system offers a practical, accurate, and compliant solution for medical institutions, improving data quality and AI integration.
- This approach addresses the limitations of existing solutions by providing a standardized, privacy-preserving, and adaptable tool for mammography reporting.
