Related Experiment Video
Updated: Jun 6, 2026
A Point-of-Care Method with Integrated Decision Support Tool to Estimate Anemia at Population Level
Published on: January 19, 2024
Comparative analysis of exit interviews and direct clinical observations in pediatric ambulatory care services in
J Onishi1, S Gupta, D H Peters
1The World Bank, Manila, Philippines. jonishi@worldbank.org
Insights
Exit interviews are unreliable for assessing healthcare provider counseling in childhood illness management. Direct observation is a more accurate method for evaluating patient communication and identifying performance gaps.
Area of Science:
- Global Health
- Pediatric Medicine
- Health Services Research
Background:
- Effective patient counseling is crucial for managing common childhood diseases.
- Assessing the quality of healthcare provider communication is essential for improving patient outcomes.
- Previous methods for evaluating counseling may lack accuracy.
Purpose of the Study:
- To compare the accuracy of receiver operating curves (ROCs) derived from direct observations versus exit interviews for counseling in pediatric care.
- To evaluate counseling practices for common childhood illnesses in Afghanistan.
Main Methods:
- A study involving 8,659 outpatient consultations for children under 5 years in Afghanistan (2005-2007).
- Direct observation using a standardized checklist served as the gold standard for counseling assessment.
- Sensitivity, specificity, and ROCs were calculated for five key counseling components using both observation and exit interviews.
Main Results:
- Counseling item prevalence was low but increasing annually (8-80%).
- Exit interviews demonstrated low sensitivity (33-88%) but higher specificity (63-91%) for counseling items.
- ROCs for exit interviews ranged from 61-77% and varied significantly by year, but not by provider characteristics.
Conclusions:
- Exit interviews are not reliable for measuring healthcare provider performance in counseling compared to direct observation.
- Direct observations revealed low but increasing counseling prevalence, highlighting significant communication gaps.
- The low accuracy of exit interviews suggests they should not be used as a standalone assessment tool for provider communication.
Objective:
To assess the receiver operating curves (ROCs) for counseling in the management of common childhood diseases comparing direct observations with exit interviews.
Design:
Eight thousand six hundred and fifty-nine randomly selected new outpatient consultations of sick children under 5 years were assessed by observation using a standardized checklist and an exit interview with their parent/guardian, taken between 2005 and 2007 from 948 health facilities in Afghanistan. The observation checklist was used as a 'gold standard' for counseling provided.
Main Measure:
Sensitivity, specificity and ROCs were estimated for five counseling items, including explanations of: a working diagnosis; what to do at home; possible adverse reactions to medicine; signs that require a return to the health facility; and a time to return.
Results:
The prevalence of counseling items was relatively low (ranging from 8 to 80%), but generally increasing each year. Exit interviews had relatively low levels of sensitivity for the counseling items, ranging from 33 to 88%, with higher levels of specificity (ranging from 63 to 91%), whereas the ROCs ranged from 61 to 77%. Although ROCs varied significantly from year to year (P < 0.002 for each item), there was little difference based on the sex or type of the health provider.
Conclusions:
Exit interviews did not provide reliable measurements of provider performance compared with direct observations. Observations identified low prevalence of counseling tasks though increasing over time. The differences between observation and exit interviews identified significant gaps in communication, suggesting that exit interviews are of low accuracy and should not be used alone.
Related Concept Videos
01:11Observational Studies
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One example of...
01:30Specialized Care Centers and Settings-II
Rural health centers are specialized care facilities in remote locations with very few medical personnel. The primary care providers who run the centers are mostly Registered Nurse Practitioners. Here, emergency treatment is provided to critically ill or injured patients before they are transferred to the closest hospital. Fortunately, due to advancement in technology, many rural healthcare facilities and professionals have easy access to diagnostic and treatment...
02:30Naturalistic Observations
01:08Data Collection by Observations
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
02:16Surveys