The Development of the Tool for Rating the qUality of Services sTandards (TRUST)

This study aimed to develop a preliminary framework for the systematic, multi-dimensional evaluation of service standards, the Tool for Rating the qUality of Service sTandards (TRUST), to address the heterogeneity and quality-related deficiencies in existing standards and to provide a preliminary evidence-informed basis for evaluating, optimizing, and promoting the quality of service standards.

TRUST was developed through an evidence based multi-step process. First, a TRUST Working Group, consisting of a drafting team, an expert panel, and a steering committee, was established. A pool of candidate items was then constructed through a systematic literature review that identified 16 existing standard evaluation frameworks, followed by a group discussion. Subsequently, a structured online expert survey was conducted to assess expert agreement on the preliminary set of items. Item weights were calculated using the Precedence Chart method. Finally, we conducted a pilot study and calculated the inter-rater reliability coefficients applying TRUST to a sample of 51 published standards.

TRUST comprises 17 items across five distinct dimensions: collaboration (28.0%), scope and purpose (7.6%), quality management (39.1%), implementation and updates (15.6%) and social impact (9.7%). Inter-rater reliability (intraclass correlation coefficient [ICC] (A,1)) across items ranged from 0.502 to 0.860, and the total score showed excellent reliability (ICC (A,1) = 0.832, 95% CI: 0.723-0.901).

TRUST offers an initial evidence-based architecture to guide the future development of assessment instruments for standards, laying the groundwork for theoretical and practical support in optimizing and refining standards.