Evaluation of UltraSTAR: performance of a collaborative structured data entry system

The UltraSTAR structured data entry system is now in routine use for reporting ultrasound studies at Brigham and Women's Hospital, having been used for 3722 reports in its first ten months of service. Reports entered through GUI-based forms are uploaded via HL7 to a radiology information system and distributed through a hospital network. UltraSTAR introduces collaborative reporting, in which nonmedical and medical staff collaborate to produce a single report for each patient visit. Performance of UltraSTAR was measured as user satisfaction, data entry time, report completeness, free text annotation rate, and referring-physician satisfaction with reports. Results show high satisfaction with UltraSTAR among radiologists and acceptance of the system among ultrasound technicians. Data entry times averaged 5.3 minutes per report. UltraSTAR reports were slightly more complete than comparable narrative reports. Free text annotations were needed in only 25.2% of all UltraSTAR reports. Referring physicians were neutral to slightly positive toward UltraSTAR's outline-format reports. UltraSTAR is successful at structured data entry despite somewhat long reporting times. Its success can be attributed to efficiencies from collaborative reporting and from integration with existing information systems. UltraSTAR shows that the advantages of structured data entry can outweigh its difficulties even before problems of data entry time and concept representation are solved.