Predictive validity of the START for unauthorised leave and substance abuse in a secure mental health setting: a pseudo-prospective cohort study

Laura E. O'Shea, Geoffrey L. Dickens

Research output: Contribution to journalArticle

5 Citations (Scopus)
10 Downloads (Pure)

Abstract

Background Risk assessment and management is central to the nursing role in forensic mental health settings. The Short Term Assessment of Risk and Treatability (START) aims to support assessment through identification of risk and protective factors. It has demonstrated predictive validity for aggression; it also aims to aid risk assessment for unauthorised leave and substance abuse where its performance is relatively untested. Objectives To test the predictive validity of the START for unauthorised leave and substance abuse. Design A naturalistic, pseudo-prospective cohort study. Settings Four centres of a large UK provider of secure inpatient mental health services. Participants Inpatients resident between May 2011 and October 2013 who remained in the service for 3-months following assessment with the START by their clinical team. Exclusion criteria were missing assessment data in excess of prorating guidelines. Of 900 eligible patients 73 were excluded leaving a final sample size of n = 827 (response rate 91.9%). Mean age was 38.5 years (SD = 16.7); most participants (72.2%) were male; common diagnoses were schizophrenia-type disorders, personality disorders, organic disorders, developmental disorders and intellectual disability. Methods Routinely conducted START assessments were gathered. Subsequent incidents of substance abuse and unauthorised leave were coded independently. Positive and negative predictive values of low and elevated risk were calculated. Receiver Operating Characteristic analysis was conducted to ascertain the predictive accuracy of the assessments based on their sensitivity and specificity. Results Patient-based rates of unauthorised leave (2.4%) and substance abuse (1.6%) were low. The positive and negative predictive values for unauthorised leave were 5.9% and 98.4%; and for substance abuse 8.1% and 99.0%. The START specific risk estimate for unauthorised leave predicted its associated outcome (Area under the curve = .659, p < .05, 95% CI .531, .786); the substance abuse risk estimate predicted its outcome with a large effect size (Area under the curve = .723, p < .01, 95% CI .568, .879). Conclusions The study provides limited support for the START by demonstrating the predictive validity of its specific risk estimates for substance abuse and unauthorised leave. High negative predictive values suggest the tool may be of most utility in screening out low risk individuals from unnecessary restrictive interventions; very low positive predictive values suggest caution before implementing restrictive interventions in those rated at elevated risk. Researchers should investigate how multidisciplinary teams formulate risk assessments for these outcomes since they outperform the quantitative element of this tool.
Original languageEnglish
Pages (from-to)970-979
Number of pages10
JournalInternational Journal of Nursing Studies
Volume52
Issue number5
DOIs
Publication statusPublished - 25 Feb 2015

Fingerprint

Substance-Related Disorders
Mental Health
Cohort Studies
Prospective Studies
Area Under Curve
Developmental Disabilities
Personality Disorders
Mental Health Services
Aggression
ROC Curve
Intellectual Disability
Sample Size
Inpatients
Schizophrenia
Nursing
Research Personnel
Guidelines
Sensitivity and Specificity

Cite this

@article{9f33cad7fd834451803601b76395a393,
title = "Predictive validity of the START for unauthorised leave and substance abuse in a secure mental health setting: a pseudo-prospective cohort study",
abstract = "Background Risk assessment and management is central to the nursing role in forensic mental health settings. The Short Term Assessment of Risk and Treatability (START) aims to support assessment through identification of risk and protective factors. It has demonstrated predictive validity for aggression; it also aims to aid risk assessment for unauthorised leave and substance abuse where its performance is relatively untested. Objectives To test the predictive validity of the START for unauthorised leave and substance abuse. Design A naturalistic, pseudo-prospective cohort study. Settings Four centres of a large UK provider of secure inpatient mental health services. Participants Inpatients resident between May 2011 and October 2013 who remained in the service for 3-months following assessment with the START by their clinical team. Exclusion criteria were missing assessment data in excess of prorating guidelines. Of 900 eligible patients 73 were excluded leaving a final sample size of n = 827 (response rate 91.9{\%}). Mean age was 38.5 years (SD = 16.7); most participants (72.2{\%}) were male; common diagnoses were schizophrenia-type disorders, personality disorders, organic disorders, developmental disorders and intellectual disability. Methods Routinely conducted START assessments were gathered. Subsequent incidents of substance abuse and unauthorised leave were coded independently. Positive and negative predictive values of low and elevated risk were calculated. Receiver Operating Characteristic analysis was conducted to ascertain the predictive accuracy of the assessments based on their sensitivity and specificity. Results Patient-based rates of unauthorised leave (2.4{\%}) and substance abuse (1.6{\%}) were low. The positive and negative predictive values for unauthorised leave were 5.9{\%} and 98.4{\%}; and for substance abuse 8.1{\%} and 99.0{\%}. The START specific risk estimate for unauthorised leave predicted its associated outcome (Area under the curve = .659, p < .05, 95{\%} CI .531, .786); the substance abuse risk estimate predicted its outcome with a large effect size (Area under the curve = .723, p < .01, 95{\%} CI .568, .879). Conclusions The study provides limited support for the START by demonstrating the predictive validity of its specific risk estimates for substance abuse and unauthorised leave. High negative predictive values suggest the tool may be of most utility in screening out low risk individuals from unnecessary restrictive interventions; very low positive predictive values suggest caution before implementing restrictive interventions in those rated at elevated risk. Researchers should investigate how multidisciplinary teams formulate risk assessments for these outcomes since they outperform the quantitative element of this tool.",
author = "O'Shea, {Laura E.} and Dickens, {Geoffrey L.}",
year = "2015",
month = "2",
day = "25",
doi = "10.1016/j.ijnurstu.2015.02.007",
language = "English",
volume = "52",
pages = "970--979",
journal = "International Journal of Nursing Studies",
issn = "0020-7489",
publisher = "Elsevier Limited",
number = "5",

}

Predictive validity of the START for unauthorised leave and substance abuse in a secure mental health setting : a pseudo-prospective cohort study. / O'Shea, Laura E.; Dickens, Geoffrey L.

In: International Journal of Nursing Studies, Vol. 52, No. 5, 25.02.2015, p. 970-979.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Predictive validity of the START for unauthorised leave and substance abuse in a secure mental health setting

T2 - a pseudo-prospective cohort study

AU - O'Shea, Laura E.

AU - Dickens, Geoffrey L.

PY - 2015/2/25

Y1 - 2015/2/25

N2 - Background Risk assessment and management is central to the nursing role in forensic mental health settings. The Short Term Assessment of Risk and Treatability (START) aims to support assessment through identification of risk and protective factors. It has demonstrated predictive validity for aggression; it also aims to aid risk assessment for unauthorised leave and substance abuse where its performance is relatively untested. Objectives To test the predictive validity of the START for unauthorised leave and substance abuse. Design A naturalistic, pseudo-prospective cohort study. Settings Four centres of a large UK provider of secure inpatient mental health services. Participants Inpatients resident between May 2011 and October 2013 who remained in the service for 3-months following assessment with the START by their clinical team. Exclusion criteria were missing assessment data in excess of prorating guidelines. Of 900 eligible patients 73 were excluded leaving a final sample size of n = 827 (response rate 91.9%). Mean age was 38.5 years (SD = 16.7); most participants (72.2%) were male; common diagnoses were schizophrenia-type disorders, personality disorders, organic disorders, developmental disorders and intellectual disability. Methods Routinely conducted START assessments were gathered. Subsequent incidents of substance abuse and unauthorised leave were coded independently. Positive and negative predictive values of low and elevated risk were calculated. Receiver Operating Characteristic analysis was conducted to ascertain the predictive accuracy of the assessments based on their sensitivity and specificity. Results Patient-based rates of unauthorised leave (2.4%) and substance abuse (1.6%) were low. The positive and negative predictive values for unauthorised leave were 5.9% and 98.4%; and for substance abuse 8.1% and 99.0%. The START specific risk estimate for unauthorised leave predicted its associated outcome (Area under the curve = .659, p < .05, 95% CI .531, .786); the substance abuse risk estimate predicted its outcome with a large effect size (Area under the curve = .723, p < .01, 95% CI .568, .879). Conclusions The study provides limited support for the START by demonstrating the predictive validity of its specific risk estimates for substance abuse and unauthorised leave. High negative predictive values suggest the tool may be of most utility in screening out low risk individuals from unnecessary restrictive interventions; very low positive predictive values suggest caution before implementing restrictive interventions in those rated at elevated risk. Researchers should investigate how multidisciplinary teams formulate risk assessments for these outcomes since they outperform the quantitative element of this tool.

AB - Background Risk assessment and management is central to the nursing role in forensic mental health settings. The Short Term Assessment of Risk and Treatability (START) aims to support assessment through identification of risk and protective factors. It has demonstrated predictive validity for aggression; it also aims to aid risk assessment for unauthorised leave and substance abuse where its performance is relatively untested. Objectives To test the predictive validity of the START for unauthorised leave and substance abuse. Design A naturalistic, pseudo-prospective cohort study. Settings Four centres of a large UK provider of secure inpatient mental health services. Participants Inpatients resident between May 2011 and October 2013 who remained in the service for 3-months following assessment with the START by their clinical team. Exclusion criteria were missing assessment data in excess of prorating guidelines. Of 900 eligible patients 73 were excluded leaving a final sample size of n = 827 (response rate 91.9%). Mean age was 38.5 years (SD = 16.7); most participants (72.2%) were male; common diagnoses were schizophrenia-type disorders, personality disorders, organic disorders, developmental disorders and intellectual disability. Methods Routinely conducted START assessments were gathered. Subsequent incidents of substance abuse and unauthorised leave were coded independently. Positive and negative predictive values of low and elevated risk were calculated. Receiver Operating Characteristic analysis was conducted to ascertain the predictive accuracy of the assessments based on their sensitivity and specificity. Results Patient-based rates of unauthorised leave (2.4%) and substance abuse (1.6%) were low. The positive and negative predictive values for unauthorised leave were 5.9% and 98.4%; and for substance abuse 8.1% and 99.0%. The START specific risk estimate for unauthorised leave predicted its associated outcome (Area under the curve = .659, p < .05, 95% CI .531, .786); the substance abuse risk estimate predicted its outcome with a large effect size (Area under the curve = .723, p < .01, 95% CI .568, .879). Conclusions The study provides limited support for the START by demonstrating the predictive validity of its specific risk estimates for substance abuse and unauthorised leave. High negative predictive values suggest the tool may be of most utility in screening out low risk individuals from unnecessary restrictive interventions; very low positive predictive values suggest caution before implementing restrictive interventions in those rated at elevated risk. Researchers should investigate how multidisciplinary teams formulate risk assessments for these outcomes since they outperform the quantitative element of this tool.

U2 - 10.1016/j.ijnurstu.2015.02.007

DO - 10.1016/j.ijnurstu.2015.02.007

M3 - Article

VL - 52

SP - 970

EP - 979

JO - International Journal of Nursing Studies

JF - International Journal of Nursing Studies

SN - 0020-7489

IS - 5

ER -