Abstract: Psychotherapy fidelity, defined as the extent to which a psychological intervention is delivered as intended, is considered to encompass both adherence (the extent to which pre-specified interventions are used) and competence (the skill with which they are implemented). While maintaining high fidelity is crucial for validating therapeutic efficacy and ensuring clinical quality, its measurement remains a significant challenge. Traditional methods, particularly the gold-standard observational coding, are hindered by high labor costs and inconsistent scoring among human raters. Alternative approaches, such as written surveys and behavioral rehearsals, fail to accurately capture real-world clinical execution. To address these issues, researchers have introduced frameworks (such as TREND, CONSORT, the BCC framework, and Carroll et al.’s framework), behavioral taxonomies (such as the Behavior Change Technique Taxonomy v1 [BCTTv1]), and machine learning methods for coding. However, these approaches remain insufficient; existing frameworks often offer broad recommendations without providing operational guidelines. Taxonomies can describe the presence of specific interventions (adherence) but tend to overlook the quality of execution (competence). Meanwhile, machine learning methods struggle with the high costs of manual data annotation and exhibit poor generalizability across different therapeutic modalities.
To address the dual challenges of the high cost and evaluation inconsistency, this paper proposes three future directions for fidelity measurement: refining measurement frameworks, establishing a unified knowledge base, and utilizing Large Language Models (LLMs) for automated assessment.
First, existing fidelity measurement frameworks can be continuously updated and refined under the guidance of evidence-based practice (EBP). A comprehensive framework should provide standardized description methods for all core dimensions of fidelity and clear rules for determining fidelity levels. To address current deficiencies in measuring therapist competence, future frameworks can integrate well-defined behavioral anchors with the Dreyfus model of skill acquisition to delineate different proficiency stages. Furthermore, grounding the framework in EBP ensures that the therapeutic components being evaluated are genuinely effective, preventing fidelity assessment from degrading into a superficial recording of clinical processes. As fidelity measurement is the prerequisite for successfully implementing evidence-based treatments, aligning these frameworks ensures that therapists effectively combine the best research evidence with clinical expertise and client characteristics.
Second, it is necessary to establish a unified fidelity measurement knowledge base to translate abstract frameworks into actionable, consistent criteria. This involves utilizing diverse strategies (such as shared component analysis and dismantling trials) to systematically extract valuable therapeutic components from various psychological treatment modalities. The paper advocates for adapting the Evidence to Decision (EtD) framework, originally used in medical guidelines, for use in psychotherapy. The EtD approach provides a systematic method for experts to review objective evidence and form consensus-based recommendations, effectively translating subjective expert experience and objective clinical evidence into unified, standardized scoring rules. This structured consensus is vital for mitigating the subjective discrepancies among expert raters.
Third, LLMs offer a technical solution to the high-cost and scalability issues of traditional human coding. Unlike traditional machine learning methods that require massive, expensive annotated datasets, LLMs possess strong few-shot learning capabilities. Given that psychotherapy is fundamentally language-based, the core task of fidelity measurement—identifying, categorizing, and quantifying verbal content and therapeutic functions—aligns with the natural language processing strengths of LLMs. Concerns regarding the inherent non-deterministic nature of LLMs can be effectively mitigated. LLMs inherently capture stable statistical regularities from massive corpora, ensuring overall consistency. Furthermore, fidelity measurement is a strictly bounded scoring task. Researchers can utilize the aforementioned structured frameworks and unified knowledge base to craft highly constrained prompts. This ensures that the evaluation boundaries, scoring dimensions, output formats, and decision rules are strictly defined. Combined with techniques like structured outputs and aggregation strategies, LLMs can provide scoring with substantially improved consistency. The efficiency and low cost of LLMs make comprehensive, large-scale quality assessments of psychotherapy processes practically feasible.
In conclusion, the future of psychotherapy fidelity measurement relies on the synergy of structured normative knowledge and advanced automated technologies. By employing EBP-guided frameworks, an EtD-based unified knowledge base, and LLM-assisted evaluation, the field can drive psychotherapy process evaluation from theoretical advocacy to a standardized, cost-effective, and highly reliable clinical practice.

Key words: fidelity, psychotherapy, psychological counseling, measurement methods, evaluation framework