Advertisement

Naturalistic outcome of nurse-led psychological therapy for mental disorders in routine outpatient care: A retrospective chart review

  • Author Footnotes
    1 Equal contribution (i.e. joint first author).
    Naoki Yoshinaga
    Correspondence
    Corresponding author.
    Footnotes
    1 Equal contribution (i.e. joint first author).
    Affiliations
    School of Nursing, Faculty of Medicine, University of Miyazaki, 5200 Kihara, Kiyotake, Miyazaki-city, Miyazaki 889-1692, Japan
    Search for articles by this author
  • Author Footnotes
    1 Equal contribution (i.e. joint first author).
    Hiroki Tanoue
    Footnotes
    1 Equal contribution (i.e. joint first author).
    Affiliations
    School of Nursing, Faculty of Medicine, University of Miyazaki, 5200 Kihara, Kiyotake, Miyazaki-city, Miyazaki 889-1692, Japan
    Search for articles by this author
  • Author Footnotes
    1 Equal contribution (i.e. joint first author).
    Yuta Hayashi
    Footnotes
    1 Equal contribution (i.e. joint first author).
    Affiliations
    Department of Nursing, Graduate School of Health Sciences, Kobe University, 7-10-2 Tomogaoka, Suma-ku, Kobe, Hyogo 654-0142, Japan
    Search for articles by this author
  • Author Footnotes
    1 Equal contribution (i.e. joint first author).
Open AccessPublished:April 27, 2022DOI:https://doi.org/10.1016/j.apnu.2022.04.008

      Highlights

      • Mental health nurses (MHNs) need to widen their skills to include effective psychotherapeutic techniques.
      • MHN-led cognitive behavioral therapy (CBT) for mental disorders is effective in routine outpatient care.
      • The finding bridges the research-practice gap in CBT utilization by MHNs.
      • The real-world evidence will encourage frontline MHNs, institutional readers, and policymakers to utilize CBT.

      Abstract

      This study aimed to investigate the clinical effectiveness of nurse-led cognitive/behavioral therapy (CBT) in Japanese routine outpatient care. We retrospectively collected data from 100 cases with mental disorders who had received nurse-led CBT. Results demonstrated that CBT provided by nurses led to significant improvements in quality of life and other clinical outcomes during the intervention period (all p < 0.001). Among participants who received optional follow-up, these improvements were well-maintained. This real-world evidence of nurse-led CBT bridges the research-practice gap, and will encourage frontline nurses and motivate institutional/organizational leaders, academic/professional societies, and policymakers to employ empirically-supported psychotherapeutic techniques in routine nursing care.

      Keywords

      Introduction

      Globally, mental illnesses/disorders are one of the greatest public health concerns, imposing a substantial burden on individuals as well as society. A significant body of evidence supports the therapeutic effectiveness of psychological therapies, particularly cognitive/behavioral therapy (CBT), across a wide array of mental health problems (e.g.
      • Butler A.C.
      • Chapman J.E.
      • Forman E.M.
      • Beck A.T.
      The empirical status of cognitive-behavioral therapy: A review of meta-analyses.
      ;
      • Wakefield S.
      • Kellett S.
      • Simmonds-Buckley M.
      • Stockton D.
      • Bradbury A.
      • Delgadillo J.
      Improving access to psychological therapies (IAPT) in the United Kingdom: A systematic review and meta-analysis of 10-years of practice-based evidence.
      ). Efforts to improve mental health services around the world have identified the need for mental health nurses (MHNs) to include empirically-supported psychotherapeutic techniques in clinical practice (e.g.
      • Hurley J.
      • Lakeman R.
      • Cashin A.
      • Ryan T.
      The remarkable (disappearing act of the) mental health nurse psychotherapist.
      ;
      • Lakeman R.
      • Cashin A.
      • Hurley J.
      • Ryan T.
      The psychotherapeutic practice and potential of mental health nurses: An Australian survey.
      ;
      • The Psychological Professions Network
      Implementing the NHS Long Term Plan: Maximising the impact of the psychological professions.
      ). As nurses represent the largest occupational group in the global health workforce, MHNs are expected to play a significant role in disseminating effective psychological therapies in mental health service.
      Although there has been discussion about the role of MHNs in providing psychotherapy (
      • Cahill J.
      • Paley G.
      • Hardy G.
      What do patients find helpful in psychotherapy? Implications for the therapeutic relationship in mental health nursing.
      ), MHNs can provide psychotherapy or use psychotherapeutic skills within their nursing frameworks. One of the earliest papers described the way that psychotherapy and nursing fit together well (
      • Schmahl J.A.
      The psychiatric nurse and psychotherapy.
      ), and some studies have also experimented with combinations of different forms of psychotherapy and several nursing models (e.g. Peplau's theory of interpersonal relations, King's theory of goal attainment, Roy's adaptation model) (
      • DeHowitt M.C.
      King's conceptual model and individual psychotherapy.
      ;
      • Freeman S.M.
      • Roy C.
      Cognitive behavior therapy and the Roy adaptation model: Integrating CBT into nursing practice.
      ;
      • Lego S.
      The application of Peplau's theory to group psychotherapy.
      ). Furthermore, recent studies in Australia identified that MHNs with psychotherapeutic skills are successfully providing uniquely holistic psychotherapeutic services; that is, bridging gaps between biomedical and psychosocial service provision to people with often complex and serious mental health problems by integrating psychotherapy into nursing practice (
      • Hurley J.
      • Lakeman R.
      • Cashin A.
      • Ryan T.
      Mental health nurse psychotherapists are well situated to improve service shortfalls in Australia: Findings from a qualitative study.
      ,
      • Hurley J.
      • Lakeman R.
      • Cashin A.
      • Ryan T.
      The remarkable (disappearing act of the) mental health nurse psychotherapist.
      ;
      • Lakeman R.
      • Cashin A.
      • Hurley J.
      • Ryan T.
      The psychotherapeutic practice and potential of mental health nurses: An Australian survey.
      ). Currently in the US, psychotherapeutic modalities are viewed as an essential skill, especially for psychiatric advanced nurse practitioners (
      American Psychiatric Nurses Association
      APNA Board of Directors endorses APNA/ISPN Joint Task Force Recommendations on the Implementation of the “Consensus Model for APRN Regulation: Licensure, Accreditation, Certification & Education”.
      ). The Chief Nursing Officer's review of mental health nursing (
      Department of Health
      From values to action: the Chief Nursing Officer’s review of mental health nursing.
      ) also recommended that MHNs need to widen their skills to include evidence-based psychotherapeutic techniques, reiterating a need for both psychotherapeutic skills and the therapeutic relationship. Furthermore, in 2012, Horatio (European Psychiatric Nurses) published a position paper about the involvement of MHNs in the practice of psychotherapy, stating that MHNs with adequate psychotherapeutic skills should be incorporated more into mental health services, especially where there are complex and severe cases that require combined psychotherapeutic and psychopharmacological support (
      Horatio: European Psychiatric Nurses
      Psychiatric/mental health nursing and psychotherapy: The position of horatio: European psychiatric nurses.
      ).
      CBT is a structured psychological intervention to help individuals improve their adaptive and emotional functioning by ascertaining and replacing inaccurate beliefs and/or maladaptive cognitions/behaviors that produce or sustain dysfunction. CBT has consistently been shown to be efficacious for various mental disorders on both a short-term and long-term basis (e.g.
      • Butler A.C.
      • Chapman J.E.
      • Forman E.M.
      • Beck A.T.
      The empirical status of cognitive-behavioral therapy: A review of meta-analyses.
      ;
      • Wakefield S.
      • Kellett S.
      • Simmonds-Buckley M.
      • Stockton D.
      • Bradbury A.
      • Delgadillo J.
      Improving access to psychological therapies (IAPT) in the United Kingdom: A systematic review and meta-analysis of 10-years of practice-based evidence.
      ). Importantly, people with mental disorders often prefer psychological treatment to pharmacological treatment (
      • McHugh R.K.
      • Whitton S.W.
      • Peckham A.D.
      • Welge J.A.
      • Otto M.W.
      Patient preference for psychological vs pharmacologic treatment of psychiatric disorders: A meta-analytic review.
      ). Nevertheless, it is often argued that service users' access to CBT is severely limited around the world, mainly due to an insufficient number of competent CBT therapists (
      • Hayashi Y.
      • Yoshinaga N.
      • Sasaki Y.
      • Tanoue H.
      • Yoshimura K.
      • Kadowaki Y.
      • Arimura Y.
      • Yanagita T.
      • Ishida Y.
      How was cognitive behavioural therapy for mood disorder implemented in Japan? A retrospective observational study using the nationwide claims database from FY2010 to FY2015.
      ;
      • Shafran R.
      • Clark D.M.
      • Fairburn C.G.
      • et al.
      Mind the gap: Improving the dissemination of CBT.
      ).
      Under these circumstances, MHNs have been engaged (or been expected to engage) in disseminating CBT in several countries. In the 1970s in the UK, MHNs became the first mental health professional group to receive a systematic CBT training primarily for neurotic disorders at the Maudsley Hospital in London (
      • Marks I.M.
      • Hallam R.
      • Connolly J.
      • Philpott R.
      Nursing in behavioral psychotherapy.
      ). The process and outcome of this initial training course have been rigorously evaluated and the data showed that utilizing trained MHNs demonstrated clinical and cost benefits in providing CBT, and they were at least as effective as other professionals (e.g. psychiatrists, psychologists) working with similar disorder groups (
      • Ginsberg G.
      • Marks I.
      Costs and benefits of behavioural psychotherapy: A pilot study of neurotics treated by nurse-therapists.
      ;
      • Marks I.M.
      Psychiatric nurse therapists in primary care: The expansion of advanced clinical roles in nursing.
      ;
      • Marks I.M.
      • Hallam R.
      • Connolly J.
      • Philpott R.
      Nursing in behavioral psychotherapy.
      ). Further, a 25-year follow-up study showed that MHNs trained in CBT had made a significant contribution to the provision of mental health service, specifically in primary care settings (
      • Gournay K.
      • Denford L.
      • Parr A.M.
      • Newell R.
      British nurses in behavioural psychotherapy: A 25-year follow-up.
      ). In other countries, Australia has instigated a systematic CBT training for MHNs in response to the need for improved service users' access to CBT (
      • Allen K.W.
      • Tolchard B.
      • Battersby M.
      Behavioural psychotherapy training for nurses in Australia: A pilot program.
      ;
      • Oakes J.
      • Tolchard B.
      • Thomas L.
      • Battersby M.
      Behavioural psychotherapy training for nurses in Australia: A trainee's view.
      ). A previous online survey with 528 Australian MHNs reported that the most popular psychotherapy currently used is CBT (nearly 70%) (
      • Fisher J.E.
      The use of psychological therapies by mental health nurses in Australia.
      ). In Japan, CBT provided by psychiatrists was initially added to the national health insurance scheme in 2010, and Japan further expanded the range of eligible CBT providers to include MHNs in 2016 in order to make CBT more available in Japanese psychiatric services.
      However, global concerns have been expressed about the utility of generalizing the results of efficacy studies to daily clinical practice (i.e. the research-practice gap) (e.g.
      • Shafran R.
      • Clark D.M.
      • Fairburn C.G.
      • et al.
      Mind the gap: Improving the dissemination of CBT.
      ). Several studies of randomized controlled trials have shown that when MHNs incorporate CBT into their practice, significant short- and/or long-term improvements in health-related outcomes occur in several mental disorders (
      • Curran J.
      • Brooker C.
      Systematic review of interventions delivered by UK mental health nurses.
      ;
      • de Pinho L.M.G.
      • Sequeira C.
      • Sampaio F.M.C.
      • Rocha N.B.
      • Ozaslan Z.
      • Ferre-Grau C.
      Assessing the efficacy and feasibility of providing metacognitive training for patients with schizophrenia by mental health nurses: A randomized controlled trial.
      ;
      • Espie C.A.
      • MacMahon K.M.
      • Kelly H.L.
      • et al.
      Randomized clinical effectiveness trial of nurse-administered small-group cognitive behavior therapy for persistent insomnia in general practice.
      ;
      • Van der Zweerde T.
      • Lancee J.
      • Slottje P.
      • Bosmans J.E.
      • Van Someren E.J.W.
      • van Straten A.
      Nurse-guided internet-delivered cognitive behavioral therapy for insomnia in general practice: Results from a pragmatic randomized clinical trial.
      ). However, most of these studies included people with mental disorders from efficacy studies (i.e. highly selective clinical trials), so they allowed only limited application to those in routine practice who have more comorbidities and receive more heterogeneous treatments. In fact, previous studies have yielded inconclusive results; some studies reported similar outcomes to randomized controlled trials, while other studies found smaller effects in routine care (e.g.
      • Hans E.
      • Hiller W.
      Effectiveness of and dropout from outpatient cognitive behavioral therapy for adult unipolar depression: A meta-analysis of nonrandomized effectiveness studies.
      ;
      • Schindler A.C.
      • Hiller W.
      • Witthöft M.
      Benchmarking of cognitive-behavioral therapy for depression in efficacy and effectiveness studies-How do exclusion criteria affect treatment outcome?.
      ). Consequently, there have been calls for research to address the effectiveness rather than the efficacy of empirically-supported psychological intervention for mental disorders in routine clinical settings (
      • Shafran R.
      • Clark D.M.
      • Fairburn C.G.
      • et al.
      Mind the gap: Improving the dissemination of CBT.
      ).
      Thus, the present study aimed to analyze the clinical effectiveness of CBT provided by MHNs under routine outpatient care conditions. In order to do so, we retrospectively collected clinical data from former cases with a wide variety of mental disorders who had received CBT delivered by MHNs in Japan.

      Materials and methods

      Design and samples collected

      We retrospectively reviewed data collected from outpatients as part of routine standard care (i.e. retrospective chart review). This study was performed in compliance with the Declaration of Helsinki, and the study protocol was reviewed and approved by the Institutional Review Board of the University of Miyazaki (approval number: O-0442). The design of this study involves no more than minimal risk and a requirement of individual informed consent would make conducting the research impracticable; hence, waiver of informed consent was granted. However, an opportunity for opt-out regarding this study was provided through posters/leaflets at the study institutions as well as institutions' websites.
      Clinical data were collected from former cases who had been diagnosed with any of the mental disorders listed in the 10th revision of the International Classification of Diseases (ICD-10 [F00-F99]) or the 4th/5th editions of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV/DSM-5), and who had received individual CBT of at least one session by an MHN at two psychiatric outpatient facilities in Miyazaki, Japan from January 1, 2016 to December 31, 2019.
      During the study period, of the 121 cases referred to MHNs by their primary psychiatrists, 112 completed the initial assessment before starting the therapy. Of these, a total of 100 cases received individual CBT of at least one session by an MHN (Fig. 1).
      Fig. 1
      Fig. 1Flow diagram of participants
      Abbreviation: CBT, cognitive behavioral therapy.

      Intervention

      In this study, we defined individual CBT as a structured psychological intervention employing an individual face-to-face format and containing components of cognitive and/or behavioral techniques (e.g. cognitive restructuring, behavioral experiments, behavioral activation, exposure, social skills training, shaping, token economy, assertion training). Since there are several standardized CBT manuals (for depression, social anxiety disorder, panic disorder, obsessive compulsive disorder, post-traumatic stress disorder, and bulimia nervosa) that have been approved by the Japanese Ministry of Health Labor and Welfare; so, when applicable, therapists followed the standardized manuals for respective disorders.
      In outpatient care in Japanese clinical settings, it is generally difficult for MHNs to secure the time and space necessary to provide 30–60 min individual sessions. However, the directors of two psychiatric outpatient facilities allowed for and arranged time and space so that the MHN-therapists could provide individual CBT within a routine care context.
      CBT was provided by three male MHNs working at the study institutions. Two of the therapists started to provide CBT from 2016, and the remaining one therapist started from 2017. At the time each therapist conducted CBT for the first time, they were aged 31.0 years (SD = 1.6), with 6.7 years of clinical experience in psychiatric care (SD = 1.1) and 2.5 years of experience in providing CBT (SD = 2.4). All therapists had completed formal CBT training in Japan (with clinical supervision), which is organized/funded by the Japanese Ministry of Health Labor and Welfare. One therapist was a certified CBT therapist of the Academy of Cognitive and Behavioral Therapies (Philadelphia, USA). On average, each therapist provided CBT for 33.3 cases (SD = 7.4) during the study period.

      Clinical outcomes

      Routine outcome data were collected at baseline (Pre), at the end of CBT (Post), and at the end of optional follow-up (FU). The primary outcome was the degree of self-reported health-related quality of life, as measured by the five-level EuroQoL five-dimensional questionnaire (EQ-5D) (
      • Shiroiwa T.
      • Ikeda S.
      • Noto S.
      • et al.
      Comparison of Value Set Based on DCE and/or TTO Data: Scoring for EQ-5D-5L Health States in Japan.
      ). Secondary outcomes were the self-reported severity of depression measured by the 9-item Patients Health Questionnaire (PHQ-9) (
      • Kroenke K.
      • Spitzer R.L.
      • Williams J.B.
      The PHQ-9: validity of a brief depression severity measure.
      ), and the self-reported severity of anxiety measured by the 7-item Generalized Anxiety Disorder scale (GAD-7) (
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Löwe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ). Furthermore, the clinicians' overall impressions of condition severity was measured by the Clinical Global Impressions-Severity (CGI-S) (
      • Guy W.
      ECDEU assessment manual for psychopharmacology.
      ). CGI-S was performed by experienced primary psychiatrists at the study institutions.

      Analysis

      All statistical analyses were based on the intention-to-treat principle. We performed the following analyses using JMP Pro Version 15.1.0 (SAS Institute Japan, Tokyo, Japan), and a two-tailed alpha level of 0.05 was used to define statistical significance.
      Among the overall sample who received CBT of at least one session, we used paired t-tests to compare the average scores for each outcome measure between the two assessment points (Pre and Post). In the primary analysis based on the intention-to-treat principle, all the dropouts were considered to be non-responders and their last observed scores were carried forward (i.e. imputing missing data). As a sensitivity analysis, paired t-tests restricted to obtained data from participants at Pre and Post (i.e. without imputing missing data) were conducted to examine the effects of missing data. However, the primary analysis based on the intention-to-treat principle present more conservative estimates of effectiveness of MHN-led CBT, so the detailed results of the sensitivity analysis are not presented here.
      Among the sub-sample who received both CBT and optional follow-up, we used a linear mixed model to evaluate the differences in the average scores for each outcome measure between the three assessment points (Pre, Post, and FU).
      The magnitude of the intervention effect was determined as the effect size based on Hedges' g. The effect sizes were categorized as follows: small (0.20–0.49), medium (0.50–0.79), and large (≥0.80).
      We also determined clinical significance based on Pre and last observed scores on PHQ-9 and GAD-7 (
      • Clark D.M.
      • Canvin L.
      • Green J.
      • Layard R.
      • Pilling S.
      • Janecka M.
      Transparency about the outcomes of mental health services (IAPT approach): An analysis of public data.
      ;
      • von Brachel R.
      • Hirschfeld G.
      • Berner A.
      • et al.
      Long-term effectiveness of cognitive behavioral therapy in routine outpatient care: A 5- to 20-year follow-up study.
      ). More detailed thresholds for clinical significance are described below:
      • ‘Recovered’ = Individuals who reliably improved (PHQ-9 reduction of ≥5 points and/or GAD-7 reduction of ≥4 points from baseline) and whose last observed scores were below the cut-off point (PHQ-9 < 10 and GAD-7 < 8).
      • ‘Improved’ = Individuals who reliably improved but whose last observed scores were above the cut-off point.
      • ‘No reliable change (unproblematic)’ = Individuals who showed no reliable change and whose last observed scores were below the cut-off point.
      • ‘No reliable change (problematic)’ = Individuals who showed no reliable change but whose last observed scores were above the cut-off point.
      • ‘Deteriorated’ = Individuals who showed reliable change in the opposite direction (PHQ-9 deterioration of ≥5 points and/or GAD-7 deterioration of ≥4 points from baseline).

      Results

      Baseline data

      Among the 100 participants who received CBT of at least one session by an MHN, the mean age was 36.5 years (SD = 10.6) and 43 participants (43%) were female. The most common primary diagnosis was major depressive disorder (n = 39), followed by social anxiety disorder (n = 15), bipolar disorder (n = 7), generalized anxiety disorder (n = 6), autism spectrum disorder (n = 5), and adjustment disorder (n = 5). The mean duration of primary diagnosis was 8.6 years (SD = 7.8). Additionally, 37 participants had comorbid diagnoses. Other baseline clinical characteristics are shown in Table 1.
      Table 1Baseline characteristics (N = 100).
      VariableValue
      Age, years, mean (SD)36.5 (10.6)
      Gender, n
       Female43
       Male57
      Marital status, n
       Single or never married60
       Married or living as married32
       Divorced8
      Employment status, n
       Full-time employment23
       Part-time employment15
       Student8
       Sick leave from work or school22
       Unemployed or homemaker32
      Primary diagnosis, n
       Depressive disordersMajor depressive disorder39
      Dysthymia1
      (total)(40)
       Anxiety disordersSocial anxiety disorder15
      Generalized anxiety disorder6
      Agoraphobia1
      (total)(22)
       Neurodevelopmental disordersAutism spectrum disorder5
      Attention deficit hyperactivity disorder3
      (total)(8)
       Bipolar and related disordersBipolar disorder7
       Trauma and stressor related disordersAdjustment disorder5
       Obsessive compulsive and related disordersObsessive compulsive disorder4
       OthersBorderline personality disorder4
      Gambling disorder3
      Schizophrenia3
      Alcohol abuse2
      Anorexia1
      Insomnia1
      (total)(14)
      Duration of primary diagnosis, years, mean (SD)11.9 (7.8)
      Comorbidity, yes, n37
      Baseline concurrent psychotropic medications, yes, n62
       Baseline antidepressant (imipramine equivalent), mg/day, mean (SD)45.7 (65.8)
       Baseline anxiolytic (diazepam equivalent), mg/day, mean (SD)7.7 (13.6)
       Baseline antipsychotic (chlorpromazine equivalent), mg/day, mean (SD)39.0 (102.7)

      Interventions received

      Of the 100 participants who received CBT of at least one session, 75 completed the course of therapy (planned CBT sessions) (dropout rate = 25%). During the intervention period (Pre-Post), on average, 11.9 sessions (SD = 7.6, range 5–31) were provided over 23.2 weeks (SD = 4.2, range 6–69). Subsequently, 48 participants who completed the course of therapy received and completed optional follow-up sessions (Fig. 1). During the optional follow-up period (Post-FU), therapists briefly reviewed individuals' progress and discussed plans for dealing with actual or potential setbacks. On average, participants received 2.8 follow-up sessions (SD = 1.8, range 1–8) over 19.5 weeks (SD = 16.3, range 2–64).
      As for concurrent psychotropic medications, no significant differences were found in the antidepressant (imipramine-equivalent), anxiolytic (diazepam-equivalent), and antipsychotic (chlorpromazine-equivalent) doses during the observation period.

      Clinical outcomes

      Table 2 presents the mean Pre and Post scores for primary and secondary outcome measures among the overall sample who received CBT (N = 100). Significant improvements were observed in all outcome measures during the intervention period (PHQ-9, GAD-7, EQ-5D, and CGI-S; all p < 0.05). The sensitivity analysis (i.e. without imputing missing data) also supported the robustness of estimated results. The effect size for EQ-5D was small (Hedges' g = 0.40), for PHQ-9 and GAD-7 were medium (both Hedges' g = 0.61), and for CGI-S was large (Hedges' g = 0.84).
      Table 2Changes in primary and secondary outcome measures among the overall sample who received CBT (Pre-Post) (N = 100).
      Mean (SD)tPEffect size (Hedges' g)
      PrePostPre-Post
      EQ-5D (primary)0.66 (0.13)0.72 (0.17)2.960.0340.40
      PHQ-913.00 (6.23)9.17 (6.30)−4.32<0.0010.61
      GAD-710.77 (5.54)7.46 (5.29)−4.32<0.0010.61
      CGI-S3.92 (0.89)3.01 (1.24)−5.84<0.0010.84
      Note: Higher scores on EQ-5D indicate better quality of life. Higher scores on the other measures indicate greater pathology or severity.
      Abbreviations: CGI-S, Clinical Global Impressions-Severity; EQ-5D, EuroQoL 5-dimensions; GAD-7, Generalized Anxiety Disorder-7 item; PHQ-9, Patient Health Questionnaire-9 item.
      Table 3 presents the mean Pre, Post, and FU scores for primary and secondary outcome measures among the sub-sample who received both CBT and optional follow-up (N = 48). Significant main effects of time were observed on all outcome measures (p < 0.001). Post-hoc pairwise comparisons indicated that significant improvements were observed on all outcome measures during the intervention period (Pre-Post) (all p < 0.001), and these improvements were maintained during the follow-up (Post-FU). The Pre-Post effect sizes for all outcome measures were large (Hedges' g = 0.87 for EQ-5D, 0.90 for PHQ-9, 0.86 for GAD-7, and 1.31 for CGI-S). Furthermore, the Pre-FU effect sizes for all outcomes were also large (Hedges' g = 1.01 for EQ-5D, 1.08 for PHQ-9, 0.93 for GAD-7, and 1.58 for CGI-S).
      Table 3Changes in primary and secondary outcome measures among the sub-sample who received both CBT and optional follow-up (Pre-FU) (N = 48).
      Mean (SD)Main effects of timeEffect size (Hedges' g)
      PrePostFUFPPre-PostPre-FU
      EQ-5D (primary)0.67 (0.11)0.79 (0.15)0.83 (0.15)15.49<0.0010.871.01
      PHQ-911.88 (5.77)6.69 (5.66)5.76 (5.49)18.75<0.0010.901.08
      GAD-710.77 (5.69)6.06 (5.12)5.64 (5.25)16.45<0.0010.860.93
      CGI-S3.88 (0.81)2.56 (1.15)2.26 (1.18)30.61<0.0011.311.58
      Note: p < 0.05, significant difference in pairwise comparisons with Pre. Higher scores on EQ-5D indicate better quality of life. Higher scores on the other measures indicate greater pathology or severity.
      Abbreviations: CGI-S, Clinical Global Impressions-Severity; EQ-5D, EuroQoL 5-dimensions; FU, Follow-up; GAD-7, Generalized Anxiety Disorder-7 item; PHQ-9, Patient Health Questionnaire-9 item.
      As for clinical significance based on self-reported scores on PHQ-9 and GAD-7, among the overall sample (N = 100), 35 participants (35.0%) were judged to be ‘recovered’ and 21 (21.0%) were ‘improved’. The remaining 44 participants (44.0%) were judged to be ‘no reliable change’ or ‘deteriorated’. Among the sub-sample who also received the optional follow-up, 25 participants (52.1%) were judged to be ‘recovered’ and 9 (18.8%) were ‘improved’, and the remaining 14 (29.2%) were judged to be ‘no reliable change’ or ‘deteriorated’. See Table 4 for more details.
      Table 4Clinical significance among the overall sample (Pre-Post, N = 100) and the sub-sample who also received optional follow-up (Pre-FU, N = 48).
      CategoryOverall sampleSub-sample
      Pre-Post (N = 100)Pre-FU (N = 48)
      n(%)n(%)
      Recovered35(35.0)25(52.1)
      Improved21(21.0)9(18.8)
      No reliable change (unproblematic)16(16.0)8(16.7)
      No reliable change (problematic)19(19.0)5(10.4)
      Deteriorated9(9.0)1(2.1)

      Discussion

      The present study aimed to investigate naturalistic clinical outcomes of MHN-led CBT for mental disorders in routine outpatient care settings, and yielded two main findings. First, CBT provided by MHNs under routine care conditions led to considerable improvements on all outcome measures during the intervention period; among participants who also received optional follow-up, these improvements were well-maintained during the follow-up. Second, more than half of participants reliably recovered/improved through receiving CBT; there was a low dropout rate, and only a few participants deteriorated. The findings listed above are also novel from a cultural perspective because, as far as we are aware, this is the first real-world evidence of MHN-led individual CBT in outpatient care settings from an Asian country.
      Our results in terms of Pre-Post/FU effect sizes (Hedges' g = 0.40–1.01 on EQ-5D and 0.61–0.84 on PHQ9/GAD-7) and clinical significance (56–70.9% recovered/improved) were comparable to other studies in routine outpatient care. As for the effectiveness of psychotherapies (including CBT) in routine care, previous studies reported within-group effect sizes of 0.89 on quality of life (Quality of Life Inventory), and 0.47–1.19 on symptom severity (e.g. Global Severity Index, PHQ-9, GAD-7) (
      • Baigent M.
      • Smith D.
      • Battersby M.
      • Lawn S.
      • Redpath P.
      • McCoy A.
      The Australian version of IAPT: clinical outcomes of the multi-site cohort study of NewAccess.
      ;
      • Cahill J.
      • Barkham M.
      • Stiles W.B.
      Systematic review of practice-based research on psychological therapies in routine clinic settings.
      ;
      • Richards D.A.
      • Borglin G.
      Implementation of psychological therapies for anxiety and depression in routine practice: Two year prospective cohort study.
      ;
      • von Brachel R.
      • Hirschfeld G.
      • Berner A.
      • et al.
      Long-term effectiveness of cognitive behavioral therapy in routine outpatient care: A 5- to 20-year follow-up study.
      ;
      • Wakefield S.
      • Kellett S.
      • Simmonds-Buckley M.
      • Stockton D.
      • Bradbury A.
      • Delgadillo J.
      Improving access to psychological therapies (IAPT) in the United Kingdom: A systematic review and meta-analysis of 10-years of practice-based evidence.
      ). Concerning levels of clinically significant change, although there are diverse definitions of recovery and improvement, previous studies in routine care reported that 11.1–58.9% of cases recovered/improved through receiving psychological therapies (
      • Cahill J.
      • Barkham M.
      • Stiles W.B.
      Systematic review of practice-based research on psychological therapies in routine clinic settings.
      ;
      • von Brachel R.
      • Hirschfeld G.
      • Berner A.
      • et al.
      Long-term effectiveness of cognitive behavioral therapy in routine outpatient care: A 5- to 20-year follow-up study.
      ). Furthermore, it is noteworthy that our study had a low dropout rate (25%), and only a few participants deteriorated (9%) during the intervention period. Taken together, in routine outpatient care settings, MHNs successfully provided CBT for various mental disorders in a satisfactory manner. However, the Pre-Post recovery rate of 35% in this study is much lower than the recovery target of 50% used in the Improving Access to Psychological Therapies (IAPT) in the UK. This is mainly because the participants in this study included more heterogeneous and severe cases (e.g. 62% received concurrent psychotropic medications at baseline) (
      • Knapstad M.
      • Nordgreen T.
      • Smith O.R.F.
      Prompt mental health care, the Norwegian version of IAPT: Clinical outcomes and predictors of change in a multicenter cohort study.
      ;
      • Richards D.A.
      • Borglin G.
      Implementation of psychological therapies for anxiety and depression in routine practice: Two year prospective cohort study.
      ;
      • von Brachel R.
      • Hirschfeld G.
      • Berner A.
      • et al.
      Long-term effectiveness of cognitive behavioral therapy in routine outpatient care: A 5- to 20-year follow-up study.
      ;
      • Werbart A.
      • Levin L.
      • Andersson H.
      • Sandell R.
      Everyday evidence: Outcomes of psychotherapies in Swedish public health services.
      ).
      In the present study, only half of the participants received optional follow-up sessions because they were not mandatory in our clinical settings. When we focus on participants who also received optional follow-up, results demonstrated that observed improvements during the intervention period were well-maintained on all outcome measures during the follow-up period. Several studies with a long-term follow-up also support our findings that individuals' clinical gains through CBT were maintained or even improved during the follow-up in routine care settings (
      • Clark D.M.
      • Layard R.
      • Smithies R.
      • Richards D.A.
      • Suckling R.
      • Wright B.
      Improving access to psychological therapy: Initial evaluation of two UK demonstration sites.
      ;
      • Hans E.
      • Hiller W.
      Effectiveness of and dropout from outpatient cognitive behavioral therapy for adult unipolar depression: A meta-analysis of nonrandomized effectiveness studies.
      ;
      • von Brachel R.
      • Hirschfeld G.
      • Berner A.
      • et al.
      Long-term effectiveness of cognitive behavioral therapy in routine outpatient care: A 5- to 20-year follow-up study.
      ). Although it is still unclear what drives the enduring effects of CBT, it is possible that participants might continue to apply cognitive/behavioral techniques after completion of the therapy (
      • Cahill J.
      • Paley G.
      • Hardy G.
      What do patients find helpful in psychotherapy? Implications for the therapeutic relationship in mental health nursing.
      ;
      • Hundt N.E.
      • Mignogna J.
      • Underhill C.
      • Cully J.A.
      The relationship between use of CBT skills and depression treatment outcome: A theoretical and methodological review of the literature.
      ). One important goal in CBT is to help clients become their own therapist, so the therapist encourages the client to integrate CBT skills and learning acquired in the session room into their daily lives. CBT's self-reliant strategies (most notably homework assignments) are as important to long-term success as self-efficacy (
      • Månsson K.N.
      • Frick A.
      • Boraxbekk C.J.
      • et al.
      Predicting long-term outcome of Internet-delivered cognitive behavior therapy for social anxiety disorder using fMRI and support vector machine learning.
      ;
      • Wilson G.T.
      • Fairburn C.C.
      • Agras W.S.
      • Walsh B.T.
      • Kraemer H.
      Cognitive-behavioral therapy for bulimia nervosa: Time course and mechanisms of change.
      ). Regarding long-term maintenance effects of CBT, the psychiatric outpatient facilities that participated in this study plan to routinely conduct a few booster or follow-up sessions at least 3–6 months after CBT completion.
      Though the present study provided valuable information, the following limitations should be considered when interpreting its findings. First, the most evident limitation is its retrospective nature, which does not allow for controlling or ruling out other confounding factors (e.g. baseline demographics, clinical variables) that might have contributed to the observed outcomes. Second, improvements through CBT observed in this study might be enhanced by potential mediators (e.g. number and duration of the intervention or follow-up). Unfortunately, we were unable to perform these analyses due to the research design and the absence of assessments at multiple time-points during the intervention period. Third, the majority of participants (69%) had mood or anxiety disorders, which resulted in limited generalizability of findings to broader psychiatric outpatient routine care. Fourth, this study had no comparison group; thus, we could not determine whether CBT itself, or other non-specific factors (including natural temporal trends), contributed to the remarkable clinical effectiveness shown in this study. As the current study was conducted within a routine care setting, this was an unavoidable limitation. Fifth, our study had a relatively small number of samples and a relatively short follow-up observation period. Only a few MHNs provided CBT in our study, mainly due to the insufficient number of competent CBT therapists in Japan (i.e. the study therapists had a sufficient experience in psychiatric nursing as well as in providing CBT). Sixth, we did not take any measures to standardize/maintain the quality of CBT (e.g. regular supervision) or check the fidelity of the intervention, mainly due to its naturalistic study design. These factors may limit the generalizability of the study findings.
      Globally, structured evidence-based psychotherapies led by MHNs are often delivered suboptimally in routine nursing care. A previous survey of Australian MHNs reported that the majority of respondents agreed that ‘Training in CBT should be made available to all MHNs’ (92%) and ‘CBT should be recommended/tried [...]’ (86%). At the same time, they reported several individual and structural factors that prevented them from employing/practicing psychotherapies in clinical practice (e.g. ‘Low morale of nurses’ [80%], ‘Lack of time within workload’ [69%], ‘Too few staff to carry out psychological therapies’ [63%]) (
      • Fisher J.E.
      The use of psychological therapies by mental health nurses in Australia.
      ). In nursing practice, a large amount of work time is generally spent on administration (including paperwork), meaning that time spent talking with clients is severely limited (
      • Barker P.
      • Buchanan-Barker P.
      Mental health in an age of celebrity: the courage to care.
      ;
      • Cameron D.
      • Kapur R.
      • Campbell P.
      Releasing the therapeutic potential of the psychiatric nurse: a human relations perspective of the nurse-patient relationship.
      ). There are also other hostile/unfavorable conditions to practicing psychological therapies both within and external to the MHN discipline, such as: (a) antipathy towards the psychotherapeutic role of the MHN (i.e. internal tensions within the MHN discipline); (b) equity of MHNs' access to funding to undertake psychotherapeutic work (e.g. minimal or no public health insurance rebate); and (c) a lack of leadership at various levels among MHNs and in management (
      • Hurley J.
      • Lakeman R.
      • Cashin A.
      • Ryan T.
      The remarkable (disappearing act of the) mental health nurse psychotherapist.
      ). For the wide-scale dissemination of CBT,
      • Gunter R.W.
      • Whittal M.L.
      Dissemination of cognitive-behavioral treatments for anxiety disorders: Overcoming barriers and improving patient access.
      proposed that practitioners and academics need to continue accumulating real-world clinical data of CBT outcomes, and then advocate and appeal for funding and organizational support.

      Conclusion

      Our results suggest that CBT provided by MHNs in psychiatric outpatient routine care settings is effective for individuals with mental disorders in improving their quality of life as well as depressive/anxiety symptoms.

      Implications for practice and policy

      Employing CBT in outpatient nursing care could give MHNs another method to use in addition to their existing nursing skills.
      • Schmahl J.A.
      The psychiatric nurse and psychotherapy.
      described how MHNs could use psychotherapeutic skills to develop better relationships with their clients. Another study demonstrated that there is a positive association between high-involvement in CBT training/practice and high levels of professional autonomy among MHNs (
      • Shiraishi Y.
      • Aoishi K.
      • Tanoue H.
      Influence of cognitive behavioral therapy on “nurse's professional autonomy” for psychiatric nurses.
      ).
      It is hoped that the positive real-world evidence of MHN-led CBT revealed through this study will encourage frontline MHNs and motivate institutional/organizational leaders, academic/professional societies, and policymakers, especially in Asian countries, to utilize evidence-based psychotherapeutic techniques in routine nursing care.

      Data availability

      Data available on request from the authors.

      Funding

      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

      CRediT authorship contribution statement

      NY contributed to the conception and design of the study. YH merged the dataset. HT analysed the data. All authors contributed to collecting the data, drafting the manuscript, critically reviewing the manuscript, and approved the final manuscript.

      Declaration of competing interest

      None.

      Acknowledgment

      The authors are grateful to Dr. Hiromitsu Fukunaga (Fukunaga Internal Medicine and Neuropsychiatry Clinic) and Dr. Kentaro Mizuno (Wakakusa Hospital) for coordinating the study, Daisuke Furushima for statistical advice, Mr. Richard White for checking the English, and Ms. Chieko Fujiyama for supporting data entry.

      References

        • Allen K.W.
        • Tolchard B.
        • Battersby M.
        Behavioural psychotherapy training for nurses in Australia: A pilot program.
        Australian and New Zealand Journal of Mental Health Nursing. 2000; 9: 75-81https://doi.org/10.1046/j.1440-0979.2000.00161.x
        • American Psychiatric Nurses Association
        APNA Board of Directors endorses APNA/ISPN Joint Task Force Recommendations on the Implementation of the “Consensus Model for APRN Regulation: Licensure, Accreditation, Certification & Education”.
        https://www.apna.org/i4a/pages/index.cfm?pageid=4495
        Date: 2011
        Date accessed: April 15, 2021
        • Baigent M.
        • Smith D.
        • Battersby M.
        • Lawn S.
        • Redpath P.
        • McCoy A.
        The Australian version of IAPT: clinical outcomes of the multi-site cohort study of NewAccess.
        Journal of Mental Health. 2020; (Advance online publication): 1-10https://doi.org/10.1080/09638237.2020.1760224
        • Barker P.
        • Buchanan-Barker P.
        Mental health in an age of celebrity: the courage to care.
        Medical Humanities. 2008; 34: 110-114https://doi.org/10.1136/jmh.2008.000539
        • Butler A.C.
        • Chapman J.E.
        • Forman E.M.
        • Beck A.T.
        The empirical status of cognitive-behavioral therapy: A review of meta-analyses.
        Clinical Psychology Review. 2006; 26: 17-31https://doi.org/10.1016/j.cpr.2005.07.003
        • Cahill J.
        • Barkham M.
        • Stiles W.B.
        Systematic review of practice-based research on psychological therapies in routine clinic settings.
        British Journal of Clinical Psychology. 2010; 49: 421-453https://doi.org/10.1348/014466509x470789
        • Cahill J.
        • Paley G.
        • Hardy G.
        What do patients find helpful in psychotherapy? Implications for the therapeutic relationship in mental health nursing.
        Journal of Psychiatric and Mental Health Nursing. 2013; 20: 782-791https://doi.org/10.1111/jpm.12015
        • Cameron D.
        • Kapur R.
        • Campbell P.
        Releasing the therapeutic potential of the psychiatric nurse: a human relations perspective of the nurse-patient relationship.
        Journal of Psychiatric and Mental Health Nursing. 2005; 12: 64-74https://doi.org/10.1111/j.1365-2850.2004.00796.x
        • Clark D.M.
        • Canvin L.
        • Green J.
        • Layard R.
        • Pilling S.
        • Janecka M.
        Transparency about the outcomes of mental health services (IAPT approach): An analysis of public data.
        Lancet. 2018; 391: 679-686https://doi.org/10.1016/s0140-6736(17)32133-5
        • Clark D.M.
        • Layard R.
        • Smithies R.
        • Richards D.A.
        • Suckling R.
        • Wright B.
        Improving access to psychological therapy: Initial evaluation of two UK demonstration sites.
        Behaviour Research and Therapy. 2009; 47: 910-920https://doi.org/10.1016/j.brat.2009.07.010
        • Curran J.
        • Brooker C.
        Systematic review of interventions delivered by UK mental health nurses.
        International Journal of Nursing Studies. 2007; 44: 479-509https://doi.org/10.1016/j.ijnurstu.2006.11.005
        • de Pinho L.M.G.
        • Sequeira C.
        • Sampaio F.M.C.
        • Rocha N.B.
        • Ozaslan Z.
        • Ferre-Grau C.
        Assessing the efficacy and feasibility of providing metacognitive training for patients with schizophrenia by mental health nurses: A randomized controlled trial.
        Journal of Advanced Nursing. 2021; 77: 999-1012https://doi.org/10.1111/jan.14627
        • DeHowitt M.C.
        King's conceptual model and individual psychotherapy.
        Perspectives in Psychiatric Care. 1992; 28: 11-14https://doi.org/10.1111/j.1744-6163.1992.tb00385.x
        • Department of Health
        From values to action: the Chief Nursing Officer’s review of mental health nursing.
        Department of Health, London2006
        • Espie C.A.
        • MacMahon K.M.
        • Kelly H.L.
        • et al.
        Randomized clinical effectiveness trial of nurse-administered small-group cognitive behavior therapy for persistent insomnia in general practice.
        Sleep. 2007; 30: 574-584https://doi.org/10.1093/sleep/30.5.574
        • Fisher J.E.
        The use of psychological therapies by mental health nurses in Australia.
        Journal of Psychiatric and Mental Health Nursing. 2014; 21: 264-270https://doi.org/10.1111/jpm.12079
        • Freeman S.M.
        • Roy C.
        Cognitive behavior therapy and the Roy adaptation model: Integrating CBT into nursing practice.
        in: Freeman S.M. Roy C. Cognitive behavior therapy in nursing practice. Springer Publishing, New York2005: 3-27
        • Ginsberg G.
        • Marks I.
        Costs and benefits of behavioural psychotherapy: A pilot study of neurotics treated by nurse-therapists.
        Psychological Medicine. 1977; 7: 685-700https://doi.org/10.1017/s0033291700006346
        • Gournay K.
        • Denford L.
        • Parr A.M.
        • Newell R.
        British nurses in behavioural psychotherapy: A 25-year follow-up.
        Journal of Advanced Nursing. 2000; 32: 343-351https://doi.org/10.1046/j.1365-2648.2000.01482.x
        • Gunter R.W.
        • Whittal M.L.
        Dissemination of cognitive-behavioral treatments for anxiety disorders: Overcoming barriers and improving patient access.
        Clinical Psychology Review. 2010; 30: 194-202https://doi.org/10.1016/j.cpr.2009.11.001
        • Guy W.
        ECDEU assessment manual for psychopharmacology.
        US Department of Health, Education, and Welfare, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, National Institute of Mental Health, Psychopharmacology Research Branch, Division of Extramural Research Programs, Rockville, MD1976
        • Hans E.
        • Hiller W.
        Effectiveness of and dropout from outpatient cognitive behavioral therapy for adult unipolar depression: A meta-analysis of nonrandomized effectiveness studies.
        Journal of Consulting and Clinical Psychology. 2013; 81: 75-88https://doi.org/10.1037/a0031080
        • Hayashi Y.
        • Yoshinaga N.
        • Sasaki Y.
        • Tanoue H.
        • Yoshimura K.
        • Kadowaki Y.
        • Arimura Y.
        • Yanagita T.
        • Ishida Y.
        How was cognitive behavioural therapy for mood disorder implemented in Japan? A retrospective observational study using the nationwide claims database from FY2010 to FY2015.
        BMJ Open. 2020; 10e033365https://doi.org/10.1136/bmjopen-2019-033365
        • Horatio: European Psychiatric Nurses
        Psychiatric/mental health nursing and psychotherapy: The position of horatio: European psychiatric nurses.
        • Hundt N.E.
        • Mignogna J.
        • Underhill C.
        • Cully J.A.
        The relationship between use of CBT skills and depression treatment outcome: A theoretical and methodological review of the literature.
        Behavior Therapy. 2013; 44: 12-26https://doi.org/10.1016/j.beth.2012.10.001
        • Hurley J.
        • Lakeman R.
        • Cashin A.
        • Ryan T.
        Mental health nurse psychotherapists are well situated to improve service shortfalls in Australia: Findings from a qualitative study.
        Australasian Psychiatry. 2020; 28: 423-425https://doi.org/10.1177/1039856220924326
        • Hurley J.
        • Lakeman R.
        • Cashin A.
        • Ryan T.
        The remarkable (disappearing act of the) mental health nurse psychotherapist.
        International Journal of Mental Health Nursing. 2020; 29: 652-660https://doi.org/10.1177/1039856220924326
        • Knapstad M.
        • Nordgreen T.
        • Smith O.R.F.
        Prompt mental health care, the Norwegian version of IAPT: Clinical outcomes and predictors of change in a multicenter cohort study.
        BMC Psychiatry. 2018; 18: 260https://doi.org/10.1186/s12888-018-1838-0
        • Kroenke K.
        • Spitzer R.L.
        • Williams J.B.
        The PHQ-9: validity of a brief depression severity measure.
        Journal of General Internal Medicine. 2001; 16: 606-613https://doi.org/10.1046/j.1525-1497.2001.016009606.x
        • Lakeman R.
        • Cashin A.
        • Hurley J.
        • Ryan T.
        The psychotherapeutic practice and potential of mental health nurses: An Australian survey.
        Australian Health Review. 2020; 44: 916-923https://doi.org/10.1071/ah19208
        • Lego S.
        The application of Peplau's theory to group psychotherapy.
        Journal of Psychiatric and Mental Health Nursing. 1998; 5: 193-196https://doi.org/10.1046/j.1365-2850.1998.00129.x
        • Månsson K.N.
        • Frick A.
        • Boraxbekk C.J.
        • et al.
        Predicting long-term outcome of Internet-delivered cognitive behavior therapy for social anxiety disorder using fMRI and support vector machine learning.
        Translational Psychiatry. 2015; 5e530https://doi.org/10.1038/tp.2015.22
        • Marks I.M.
        Psychiatric nurse therapists in primary care: The expansion of advanced clinical roles in nursing.
        Royal College of Nursing, London1985
        • Marks I.M.
        • Hallam R.
        • Connolly J.
        • Philpott R.
        Nursing in behavioral psychotherapy.
        Royal College of Nursing, London1977
        • McHugh R.K.
        • Whitton S.W.
        • Peckham A.D.
        • Welge J.A.
        • Otto M.W.
        Patient preference for psychological vs pharmacologic treatment of psychiatric disorders: A meta-analytic review.
        Journal of Clinical Psychiatry. 2013; 74: 595-602https://doi.org/10.4088/JCP.12r07757
        • Oakes J.
        • Tolchard B.
        • Thomas L.
        • Battersby M.
        Behavioural psychotherapy training for nurses in Australia: A trainee's view.
        International Journal of Mental Health Nursing. 2002; 11: 139-141https://doi.org/10.1046/j.1440-0979.2002.00238.x
        • Richards D.A.
        • Borglin G.
        Implementation of psychological therapies for anxiety and depression in routine practice: Two year prospective cohort study.
        Journal of Affective Disorders. 2011; 133: 51-60https://doi.org/10.1016/j.jad.2011.03.024
        • Schindler A.C.
        • Hiller W.
        • Witthöft M.
        Benchmarking of cognitive-behavioral therapy for depression in efficacy and effectiveness studies-How do exclusion criteria affect treatment outcome?.
        Psychotherapy Research. 2011; 21: 644-657https://doi.org/10.1080/10503307.2011.602750
        • Schmahl J.A.
        The psychiatric nurse and psychotherapy.
        Nursing Outlook. 1962; 10: 460-465
        • Shafran R.
        • Clark D.M.
        • Fairburn C.G.
        • et al.
        Mind the gap: Improving the dissemination of CBT.
        Behaviour Research and Therapy. 2009; 47: 902-909https://doi.org/10.1016/j.brat.2009.07.003
        • Shiraishi Y.
        • Aoishi K.
        • Tanoue H.
        Influence of cognitive behavioral therapy on “nurse's professional autonomy” for psychiatric nurses.
        Journal of Japan Academy of Psychiatric and Mental Health Nursing. 2014; 23 ([in Japanese]): 58-64https://doi.org/10.20719/japmhn.KJ00009702525
        • Shiroiwa T.
        • Ikeda S.
        • Noto S.
        • et al.
        Comparison of Value Set Based on DCE and/or TTO Data: Scoring for EQ-5D-5L Health States in Japan.
        Value in Health. 2016; 19: 648-654https://doi.org/10.1016/j.jval.2016.03.1834
        • Spitzer R.L.
        • Kroenke K.
        • Williams J.B.
        • Löwe B.
        A brief measure for assessing generalized anxiety disorder: the GAD-7.
        Archives of Internal Medicine. 2006; 166: 1092-1097https://doi.org/10.1001/archinte.166.10.1092
        • The Psychological Professions Network
        Implementing the NHS Long Term Plan: Maximising the impact of the psychological professions.
        • Van der Zweerde T.
        • Lancee J.
        • Slottje P.
        • Bosmans J.E.
        • Van Someren E.J.W.
        • van Straten A.
        Nurse-guided internet-delivered cognitive behavioral therapy for insomnia in general practice: Results from a pragmatic randomized clinical trial.
        Psychotherapy and Psychosomatics. 2020; 89: 174-184https://doi.org/10.1159/000505600
        • von Brachel R.
        • Hirschfeld G.
        • Berner A.
        • et al.
        Long-term effectiveness of cognitive behavioral therapy in routine outpatient care: A 5- to 20-year follow-up study.
        Psychotherapy and Psychosomatics. 2019; 88: 225-235https://doi.org/10.1159/000500188
        • Wakefield S.
        • Kellett S.
        • Simmonds-Buckley M.
        • Stockton D.
        • Bradbury A.
        • Delgadillo J.
        Improving access to psychological therapies (IAPT) in the United Kingdom: A systematic review and meta-analysis of 10-years of practice-based evidence.
        British Journal of Clinical Psychology. 2021; 60: 1-37https://doi.org/10.1111/bjc.12259
        • Werbart A.
        • Levin L.
        • Andersson H.
        • Sandell R.
        Everyday evidence: Outcomes of psychotherapies in Swedish public health services.
        Psychotherapy. 2013; 50: 119-130https://doi.org/10.1037/a0031386
        • Wilson G.T.
        • Fairburn C.C.
        • Agras W.S.
        • Walsh B.T.
        • Kraemer H.
        Cognitive-behavioral therapy for bulimia nervosa: Time course and mechanisms of change.
        Journal of Consulting and Clinical Psychology. 2002; 70: 267-274