Lessons learned in evaluating system interdependencies using qualitative methods

Area of Science:

• Clinical and Translational Science
• Systems Science in Healthcare
• Research Evaluation Methodologies

Background:

• Clinical and Translational Research (CTR) centers face complex system interdependencies.
• Evaluating these interdependencies is crucial for optimizing research processes and outcomes.
• Traditional evaluation methods may not effectively capture the nuances of system dynamics with diverse stakeholders.

Purpose of the Study:

• To share practical lessons learned from evaluating system interdependencies within a CTR.
• To address the methodological challenges of discussing complex system concepts with layperson participants.
• To inform future evaluation strategies for complex research environments.

Main Methods:

• Exploration of system interdependencies within a clinical and translational research center.
• Qualitative data collection through interviews with layperson participants.
• Iterative refinement of interview approaches, moving from structured to open narrative methods.

Main Results:

• Discussing abstract system concepts (e.g., cascading failures, feedback loops) with non-experts presents significant methodological challenges.
• An open narrative interview approach proved more effective for eliciting rich data on system interdependencies from layperson participants.
• Adaptability in evaluation methodology is key to understanding complex organizational dynamics.

Conclusions:

• Effective evaluation of system interdependencies in CTRs necessitates tailored communication strategies.
• Shifting towards more flexible, narrative-based data collection enhances participant engagement and data quality.
• Lessons learned emphasize the importance of methodological innovation in research center evaluations.