Related Experiment Video
Updated: Jun 13, 2025
Modeling Breast Cancer in Human Breast Tissue using a Microphysiological System
Published on: April 23, 2021
Assessing the Reliability and Validity of Principles for Health-Related Information on Social Media (PRHISM) for
Hiroki Kusama1, Yoshimitsu Takahashi2, Shunichiro Orihara3
1Department of Breast Surgical Oncology, Tokyo Medical University Hospital, 6-7-1, Nishishinjuku, Shinjuku-ku, Tokyo, 160-0023, Japan, 81 3-3342-6111.
Background:
There is breast cancer-related medical information on social media, but there is no established method for objectively evaluating the quality of this information. Principles for Health-Related Information on Social Media (PRHISM) is a newly developed tool for objectively assessing the quality of health-related information on social media; however, there have been no reports evaluating its reliability and validity.
Objective:
The purpose of this study was to statistically examine the reliability and validity of PRHISM using videos about breast cancer treatment on YouTube (Google).
Methods:
In total, 60 YouTube videos were selected on January 5, 2024, with the Japanese words for "breast cancer," "treatment," and "chemotherapy," and assessed by 6 Japanese physicians with expertise in breast cancer. These evaluators independently evaluated the videos using PRHISM and an established tool for assessing the quality of health-related information, DISCERN, as well as through subjective assessments. We calculated interrater and intrarater agreement among evaluators with CIs, measuring agreement using weighted Cohen kappa.
Results:
The interrater agreement for PRHISM overall quality was κ=0.52 (90% CI 0.49-0.55), indicating that the expected level of agreement, statistically defined by the lower limit of the 90% CI exceeding 0.53, was not achieved. However, PRHISM demonstrated higher agreement compared with DISCERN overall quality, which had a κ=0.45 (90% CI 0.41-0.48). In terms of validity, the intrarater agreement between PRHISM and subjective assessments by breast experts was κ=0.37 (95% CI 0.14-0.60), while DISCERN showed an agreement of κ=0.27 (95% CI 0.07-0.48), indicating fair agreement and no significant difference in validity.
Conclusions:
PRHISM has demonstrated sufficient reliability and validity for evaluating the quality of health-related information on YouTube, making it a promising new metric. To further enhance objectivity, it is necessary to explore the use of artificial intelligence and other approaches.
