Related Experiment Video
Updated: Jun 25, 2026
12:04
Assessment of the Cytotoxic and Immunomodulatory Effects of Substances in Human Precision-cut Lung Slices
Published on: May 9, 2018
13.8K
ChatGPT in Occupational Medicine: A Comparative Study with Human Experts
Martina Padovan1, Bianca Cosci1, Armando Petillo1
1Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, 56126 Pisa, Italy.
Bioengineering (Basel, Switzerland)
|January 22, 2024
Summary
ChatGPT's accuracy in occupational health medicine was evaluated. While AI-generated answers with legislative context were comparable to physicians, human professionals were preferred for complex medical questions.
Area of Science:
- Medical Informatics
- Occupational Health
- Artificial Intelligence
Background:
- The integration of Artificial Intelligence (AI) into healthcare, particularly in specialized fields like occupational health medicine, presents both opportunities and challenges.
- Evaluating the accuracy and reliability of AI tools, such as ChatGPT, is crucial for understanding their potential impact on medical practice and decision-making.
Purpose of the Study:
- To assess the accuracy and reliability of ChatGPT in responding to complex occupational health medical queries.
- To explore the implications and limitations of AI in the domain of occupational health medicine.
- To offer recommendations for future research and inform stakeholders on AI's role in healthcare.
Main Methods:
- A dataset of occupational medicine questions and answers based on Italian legislation was developed by a panel of physicians.
- ChatGPT generated answers to these questions, both with and without legislative context.
- Physicians, divided into two teams, blindly evaluated human-generated and AI-generated answers, with each team reviewing the other's work.
Main Results:
- Occupational physicians demonstrated superior performance in formulating accurate questions compared to ChatGPT, based on a 5-point Likert scale.
- ChatGPT's answers, when provided with legislative context, were found to be comparable in quality to those generated by professional doctors.
- Despite comparable accuracy, a discernible user preference for human-generated answers was observed, highlighting the continued value placed on professional medical expertise.
Conclusions:
- AI tools like ChatGPT show promise in providing accurate medical information within specific contexts, such as occupational health legislation.
- Human oversight and professional judgment remain critical, as users express a preference for answers from occupational medicine professionals.
- Further research is needed to fully understand and optimize the role of AI in occupational health medicine, balancing technological capabilities with human expertise.
