Measuring recovery in participants with schizophrenia spectrum disorder: validation of the individual recovery score counter (I.ROC). | BMC Psychiatry

by admin
Measuring recovery in participants with schizophrenia spectrum disorder: validation of the individual recovery score counter (I.ROC). | BMC Psychiatry

Translation of I.ROC

I.ROC was translated from English to Dutch by a group of experienced researchers, practitioners, participants and experts. It was back-translated by NST Science (, an independent translation agency, and then presented to the developers for comments, resulting in some corrections. The discussion points were presented to the research team and the translation agency, after which the final version was approved by the original authors. Translation guidelines as suggested by Van Widenfelt et al. [28] were followed.

Participants and procedure

The study was conducted from June 2016 to December 2018 in Flexible Assertive Community Treatment Teams (FACT). [29] in four field mental health services; GGZ Drenthe Mental Health Institute, GGZ Friesland Mental Health Care Service, Lentis Psychiatric Institute and GGz Breburg Mental Health Institute. Inclusion criteria were: age 18 to 65 years, diagnosed with a schizophrenia spectrum disorder as determined by a psychiatrist or psychologist, able to give written informed consent, sufficient Dutch language proficiency, receiving care for at least one year.

A sample size ≥ 7 is required to assess content validity [30]. Therefore, seven participants with schizophrenia spectrum disorder were invited to participate by their clinicians. To assess the other psychometric properties of the I.ROC, participants with schizophrenia spectrum disorder who were invited for an annual routine outcome assessment [31], were invited to participate. Based on Clark and Watson [32]a sample size of ≥ 300 is required and the COSMIN criteria assume at least 7 participants per question [30]both criteria were met in the present study.

Eligible participants were informed about the study procedures and then asked for written informed consent. The Medical Ethics Committee (METC) of the University Medical Center Groningen concluded that the evaluation with I.ROC falls outside the scope of the Medical Research Involving Human Subjects (WMO) Act (2016-02-23, number M16.188934). The study was approved by the local scientific committees of all four participating mental health institutes.

As a first step in the ongoing validation, service users were asked for their views on the tool. To prevent socially desirable responses, two other reinstatement measures (RAS [20]and the NHS [33]) were added. Participants were asked about the appropriateness and comprehensiveness of recovery measures. The order of measures presented varies for each participant.

In the second phase of validation testing, participants were assessed at baseline (t0), after six months (t1) and after twelve months (t2) with the I.ROC and a set of additional questionnaires; PANSS [17]FR tool [21, 34]HoNOS [19]MANSA [18]and RAS [20] to assess validity and sensitivity to change. To assess test–retest reliability, participants completed the I.ROC twice, fourteen days apart with the same rater under the same conditions (eg, time of day, day of week). To ensure a robust test–retest protocol, participants had to remain stable; this was overseen by their case manager. Data collection was performed by trained nurses, undergraduate nursing students and experienced experts.


The Individual recovery results counter [24] includes 12 items rated on a 6-point Likert scale from 1 (never) to 6 (all the time). Higher scores reflect greater progress toward personal recovery. For a more detailed description, see above.

The Positive and Negative Syndrome Scale (PANSS [17]) is a typological and dimensional assessment tool for psychotic symptoms and has been evaluated by clinicians. The PANSS consists of 30 items rated on a 7-point Likert scale from 1 (absent) to 7 (extreme), with higher scores indicating more symptoms. The PANSS consists of three subscales; positive symptoms, negative symptoms, and general psychopathology. PANSS ratings are based on a semi-structured interview. Psychometric properties appear good; internal consistency was acceptable (α = 0.79 [17]).

The Functional Remission Tool (FR instrument [21, 34]) is a 3-item instrument assessing social recovery in people with severe mental illness in three domains: living and self-care, work and learning, and social contacts. The FR instrument is a clinician-rated semi-structured interview with the patient or a close other. Scores range from 0 to 2, with higher scores indicating less remission. Psychometric properties were assessed; internal consistency was acceptable (α = 0.70 [21, 34]).

The Results for the health of the nation (HoNOS [19]) is an instrument for assessing the client’s mental and social functioning. The HoNOS is rated by clinicians and consists of twelve items, scores ranging from 0 (no problem) to 4 (very severe problem). A review of the psychometric properties of the HoNOS showed internal consistency ranging from 0.59 to 0.76 [35].

The MANchester Brief Assessment of Quality of Life (MANSA [18]) is a quality of life assessment tool focused on satisfaction with life in general and with different domains of life, including physical and mental health. This self-report questionnaire contains twelve items rated on a 7-point Likert scale. Scores range from 0 (very dissatisfied) to 6 (very satisfied). The psychometric properties of the MANSA have been tested and internal consistency is acceptable (α = 0.74 [18] to good (α = 0.81 [36]).

The Recovery Rating Scale (RAS [20]) is a self-report personal recovery questionnaire. The original version consists of 41 questions, but shorter versions are known. We used the RAS-24 to assess the convergent validity of the I.ROC. Items are rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). High scores indicate more recovery. A review of the psychometric properties of the RAS [37] described that internal consistency across studies was acceptable to excellent (α = 0.76-0.97). The review reported significant positive correlations with the RAS and measures of quality of life, meaning in life, empowerment, self-esteem, sense of significance, and hope.

Analysis plan

To assess content validity, we conducted a qualitative pilot study using a semi-structured interview by an experienced interviewer. Interviews were recorded and transcribed verbatim and analyzed by two trained researchers using ATLAS.ti 8 Windows.

Internal consistency of I.ROC and the two main factors (intrapersonal and interpersonal [24]) were calculated using Cronbach’s alpha, with α ≥ 0.70 as acceptable. Test–retest reliability of the I.ROC was analyzed by measuring the strength of correlation and concordance between the two I.ROC scores fourteen days apart using the Pearson correlation coefficient and the intraclass correlation coefficient (ICC; model two-way random, type sequence [38, 39]). Values ​​equal to or greater than 0.70 are considered acceptable. Pearson’s correlation coefficient was used to calculate convergent validity. Coefficients of 0.10 to 0.39 were considered weak, 0.40–0.59 moderate, and correlations of 0.60 or greater strong. We expected a moderate correlation between the I.ROC and the comparison measures because most of them measured only one or two domains of recovery and were reported by clinicians rather than self-reported.

Sensitivity to change over time was assessed by comparing scores at baseline (t0), six months (t1) and 12 months (t2) using one-way repeated measures ANOVA as suggested by Stratford & Riddle [40]. The difference between I.ROC total scores at t0 and t1 was compared with the difference in RAS and MANSA total score at t0 and t1, using Pearson’s correlation coefficient to measure the strength of correlation. Data were analyzed with SPSS version 23 for Windows. We expected very little change between time points because recovery in all three domains for people with schizophrenia spectrum disorder often takes years [14].

A confirmatory factor analysis was conducted on categorical data using multidimensional item response theory to examine whether the twelve questions could be separated into the two underlying dimensions as suggested by Monger et al. [24]or should be treated as a single factor, as suggested by Dickens et al. [26]. We compared the two-factor model with the one-factor model and with the original four-factor model, which was based on the HOPE model of the I.ROC. Appropriate indices were selected to test which model best represented the present data set [41]: root mean square error of approximation (RMSEA; cutoff close to 0.06), Tucker-Lewis index (TLI; cutoff close to 0.90, higher is better) and comparative fit index (CFI ; cut-off value close to 0.90, higher is better). CFA data were analyzed in R (version 3.6.0; using the lavaan package for structural equation modeling [42].

Source Link

You may also like