ORIGINAL ARTICLE

Original article Arch Argent Pediatr 2019;117(5):301-305/301

http://dx.doi.org/10.5546/aap.2019.eng.301

Psychometric properties and operating characteristics of the Spanish-adapted version of the CRAFFT questionnaire in adolescents from Greater Buenos Aires

 

David Colica, M.D.^a, Verónica Campana, M.D.^a, Fernando R. Vázquez Peña, M.D.^a, Pascual Barán Cegla, M.D.^a and Valeria Vietto, M.D.^a

a. Family and Community Medicine Service of Hospital Italiano de Buenos Aires, Autonomous City of Buenos Aires, Argentina.

E-mail address: Valeria Veto, M.D.: valeria.vietto@hospitalitaliano.org.ar

Funding: This study was funded through a research fellowship granted by the Mutual Association of Professionals of Hospital Italiano (Asociación Mutual de Profesionales del Hospital Italiano, AMPHI). The AMPHI did not have any participation in the study design, data collection, analysis, interpretation, manuscript writing or decision to send it for publication.

Conflict of interest:None.

Received: 9-21-2018
Accepted:3-31-2019

 

---

ABSTRACT

Introduction: The Car, Relax, Alone, Forget, Family and Friends, Trouble (CRAFFT) questionnaire is an instrument used in the screening for problematic alcohol and substance use, abuse or dependence in adolescents. While there is a Spanish-adapted and validated version for Argentine adolescents (CRAFFTa), it cannot be applied indiscriminately because it has not been assessed in different sociocultural settings. Objective: To assess the transferability of the CRAFFTa as a screening tool for problematic alcohol and drug use, abuse or dependence in an adolescent population of low socio-economic level.
Population and methods: An anonymous survey was administered to low-income adolescents living in Greater Buenos Aires and attending two secondary schools. The Spanish-adapted version of the CRAFFT questionnaire, validated for its use in Argentina, was administered, while the Problem Oriented Screening Instrument for Teenagers, substance use and abuse subscale (POSITuas) questionnaire was used as a reference test.
Results: A total of 146 questionnaires were completed (completion rate: 89.6 %). Reliability was 0.681; criterion validity: 0.697; sensitivity: 93.4; specificity: 72.5; precision: 0.886; positive predictive value: 0.898; negative predictive value: 0.966. Compound reliability was 0.872; average variance extracted: 0.718. Convergent validity was acceptable in the confirmatory factor analysis, and a single-factor structure was used.
Conclusions: The operating characteristics of the CRAFFTa questionnaire are acceptable for its use in the screening for problematic alcohol and substance use, abuse or dependence in a low-income adolescent population.

Key words: Adolescent; Screening; Alcohol use in adolescents; Substance abuse screening; Substance-related disorders.

---

 

INTRODUCTION

The initiation of psychoactive substance use during adolescence is associated with a higher risk for persistent substance use, abuse, dependence, and other comorbid disorders at a later stage in life.^1,2

The most commonly used drugs among 14 to 18-year-old students include alcohol, tobacco, cannabis, and tranquilizers or sleeping pills.^3-6 In Argentina, in 2014, approximately 70 % of adolescents reported that they had used alcohol at least once, whereas 67 %, 36 %, and 16 % said they had tried energy drinks, tobacco, and marihuana, respectively.⁷

It has been documented that using structured screening tools during primary health care visits improves the detection of substance use problems in adolescents.⁸

The Car, Relax, Alone, Forget, Family and Friends, Trouble (CRAFFT) questionnaire is used to assess the risk for problematic alcohol and substance use, abuse or dependence in adolescents. Its acronym stands for the 6 key words that make up the questionnaire.⁹The instrument was translated and adapted into Spanish in Argentina,¹⁰ and its performance was correlated to another instrument, the Problem Oriented Screening Instrument for Teenagers, substance use and abuse subscale (POSITuas).^11-13

However, given that the sample included in the validation study was representative of the health subsystems because participants had union or private health insurance plans and a high level of education compared to the national average, it is unknown whether the Argentina version (CRAFFTa) maintains its psychometric properties in different sociocultural settings.

The objective of this study was to assess the transferability of the CRAFFTa as a screening tool for problematic alcohol and drug use in an adolescent population of low socio-economic level.

POPULATION AND METHODS

We conducted an observational, crosssectional, analytical study.

The target population corresponded to adolescents aged 11-19 years attending two secondary schools in the city of Boulogne Sur Mer, district of San Isidro, province of Buenos Aires, in November 2016. Individuals who had a diagnosis of developmental delay, mental disability or psychiatric disorder that undermined their autonomy to voluntarily decide to participate in the study were excluded.

Although the Family and Community Medicine Center San Pantaleon covered 4 secondary schools, only those where the health team conducted community outreach activities for the promotion and prevention of health aimed at children and adolescents were selected, and the respective education authorities were asked for their approval.

The transferability of the CRAFFTa in the studied population was assessed using the POSITuas as a reference test (Annexes 1 and 2) (Spanish version).¹⁰ In both cases, a score of 1 was assigned to each "yes" answer, and 0 to each "no" answer. Individuals were considered to have a risk for problematic substance use, abuse or dependence if they scored a total of 2 or more, respecting the cut-off point established by the original authors of both instruments.

For data collection, all students who attended each school on a specific, arbitrarily selected date during the corresponding school year were invited to participate. The physicians of the research team went into the classrooms during school hours and delivered the printed tools in an opaque envelope to each participant, after obtaining their oral informed consent. In order to warrant participants' confidentiality, privacy, and autonomy, and in agreement with the Convention on the Rights of the Child, National Law N.° 26,061 for the Comprehensive Protection of Children and Adolescents, and National Law N.° 26,529 for Patient Rights, Medical Records, and Informed Consents, questionnaires were completed in an anonymous manner and leaving out identifying data. In accordance with guideline 17 of the Council for International Organizations of Medical Sciences (CIOMS),¹⁴ parents' authorization for study participation was waived; the study protocol was approved by the Protocol Ethics Committee of Hospital Italiano de Buenos Aires on July 28^th, 2016 (number 2863).

Reliability (KR-20 coefficient),¹⁵ external criterion validity (Pearson correlation),¹⁶ sensitivity and specificity, precision (Receiver Operating Characteristic [ROC] curve), negative predictive value (NPV) and positive predictive value (PPV) were assessed (considering a 72.2 % prevalence of alcohol use, as reported by the national government for the province of Buenos Aires),¹⁷ using the GNU PSPP software and comparing each participant's answers to both questionnaires.

In addition, the construct validity was assessed using a confirmatory factor analysis (CFA), which estimated compound reliability and average variance extracted as expressions of reliability, and convergent validity, for which the answers to the CRAFFTa questionnaire were used. This analysis was done using the Mplus software, based on a single-factor structure^18-21 and the weighted least squares means and variance adjusted (WLSMV) estimator for dichotomous outcome measures.²²

Participants were selected by consecutive sampling. According to the formula proposed by Joreskog,²³ [(k + 1)(k + 2)]/2, where k means the number of indicators, 28 observations were necessary to establish an asymptotic covariance matrix and perform the CFA of the 6 questionnaire items. However, for conservative purposes and considering the most recent bibliography,²² a sample size of at least 150 surveys was proposed.

RESULTS

Out of 270 students enrolled in the schools where the study was conducted, 163 adolescents were present in their classrooms and were invited to participate. After excluding cases who had left at least one question unanswered, 151 valid CRAFFTa questionnaires were obtained for the CFA, together with 146 valid CRAFFTa and POSITuas questionnaire pairs for the analysis of the rest of the psychometric properties (completion rate: 89.6 %). A score of 2 or more was obtained by 43 % (63) (95 % confidence interval [CI]: 20-35) of adolescents in the CRAFFTa, and by 27 % (40) (95 % CI: 35-52) in the POSITuas.

The reliability of the CRAFFTa, as measured by the KR-20 coefficient, was 0.681, while its criterion validity was 0.697. Compared to the POSITuas, the instrument's sensitivity was 93.4 %, and its specificity, 72.5 %, with a precision of 0.886, given by the ROC curve (95 % CI: 0.8180.954). The PPV was 0.898 and the NPV, 0.966.

The results of overall fit indices of the CFA were adequate (see Table 1). The compound reliability was 0.872 (recommended: higher than 0.7)¹⁵ and the average variance extracted, 0.718 (recommended: higher than 0.5)¹⁵. The convergent validity was acceptable because 5 standardized factor loadings showed values higher than recommended (higher than 0.5 and, ideally, higher than 0.7), and only 1 was within the recommended limit, with a 0.47 value.²³ All factor loadings were statistically significant (p < 0.00001) (Figure 1).

Table 1. Overall fit indices of confirmatory factor analysis

Figure 1. Confirmatory factor analysis. It describes the standardized factor loadings using the weighted least square means and variance adjusted method

DISCUSSION

This study tested the transferability of the CRAFFTa questionnaire in adolescents who attended school in a low socio-economic neighborhood in Greater Buenos Aires.

Comparing answers to the POSITuas questionnaire, the CRAFFTa showed an acceptable external criterion validity (0.697) and an adequate precision (0.886).

Likewise, it demonstrated an adequate performance in its operating characteristics. The sensitivity was 93.4 %, which was higher than that reported by the validation studies for the original and reference instrument (76 % and 59 %, respectively). Although specificity was 72.5 %, which was slightly lower than that obtained in other studies (94 % and 88 %, respectively), and considering the purpose of the instrument, the values observed here were deemed adequate and supported its use as a screening test.

In addition, considering the setting of a high prevalence of psychoactive substance use where our study was conducted,¹⁷ the PPV (0.898) and NPV (0.966) were excellent. It is worth noting that, although it would have been ideal to use the prevalence of alcohol or marihuana or other drug use to establish the instrument's predictive values, such combined information was not available because the official numbers are reported for each substance separately.¹⁷ The prevalence of alcohol use was applied for the analysis because it was the one that showed more indicators of use among adolescents.

In relation to the reliability of the CRAFFTa, the estimations obtained with both methods showed adequate values (0.681 for the KR-20 coefficient; 0.872 for the compound reliability, and 0.718 for the average variance extracted in the CFA). Prior studies had documented an internal consistency using Cronbach's alpha (varying between 0.64 and 0.74)^9,18,24 or the KR-20 coefficient (0.74).¹⁹ In addition, the successful CFA results were consistent with those carried out by previous authors, thus confirming the instrument's single-factor structure.^18-21

A possible limitation of this study is that data were collected from self-reports, which may not reflect actual substance use by survey respondents and may result in an information bias because of fear to punishment or reprimand about the behavior under study. For this reason, anonymity of answers was warranted by using closed, opaque envelopes and leaving out all identifying data and, in accordance with the CIOMS guidelines, parents' authorization was waived, thus safeguarding participants from the possibility of being questioned or intimidated. Prior studies also documented that self-reported alcohol and/or psychoactive substance use was a reliable measure in clinical and school settings.²⁵

One of the strengths of this study was that the sample of adolescents was obtained in the school setting because that is where public and primary health care actions are planned in the framework of primary care practice aimed at the community.²⁶

In addition, and given that the instrument's validity had been tested in Argentina in adolescents from a middle- and high-socioeconomic level, the results of this study support its use as a screening instrument also in primary health care centers (PHCCs) located in low-resource communities, which will allow for the detection of cases that require individualized interventions.

CONCLUSION

The psychometric properties of the CRAFFTa questionnaire are acceptable to support the use of this instrument in the screening for problematic alcohol and substance use in the adolescent population with a low socio-economic level.

ANNEX 1.

CUESTIONARIO DE PESQUISA DE PROBLEMAS EN ADOLESCENTES (POSITuas)

El propósito de estas preguntas es ayudarnos a conocer la mejor manera de ayudarte. Por esto, trata de responderlas con franqueza.

Este no es un examen; no hay respuestas correctas o incorrectas. Todas las respuestas serán confidenciales.

Contesta todas las preguntas. Si alguna de ellas no se aplica exactamente a ti, escoge la respuesta que más se acerque a la verdad en tu caso. Por favor, marca una "X" sobre tu respuesta.

Si no comprendes alguna palabra, pide ayuda a la persona encargada.

¡Gracias!

1.	¿Te metes en problemas porque consumes drogas o bebidas alcohólicas en la escuela?	Sí_	No_
2.	¿Te has hecho daño o le has hecho daño a otra persona accidentalmente estando bajo los efectos del alcohol o de drogas?	Sí_	No_
3.	¿A veces, no puedes participar en actividades porque te has gastado el dinero en drogas o bebidas alcohólicas?	Sí_	No_
4.	¿Sientes, a veces, que eres adicto(a) al alcohol o las drogas?	Sí_	No_
5.	¿Has comenzado a consumir mayor cantidad de alcohol o drogas para obtener el efecto que deseabas?	Sí_	No_
6.	¿Te vas, a veces, de alguna fiesta porque no hay bebidas alcohólicas o drogas?	Sí_	No_
7.	¿Sientes un deseo constante de consumir bebidas alcohólicas o drogas?	Sí_	No_
8.	¿Has tenido un accidente automovilístico estando bajo el efecto del alcohol o de drogas?	Sí_	No_
9.	¿Se te olvidan las cosas que hiciste mientras estabas consumiendo alcohol o drogas?	Sí_	No_
10.	Durante el mes pasado, ¿has manejado un automóvil estando borracho(a) o drogado(a)?	Sí_	No_
11.	¿El uso de alcohol o drogas te produce cambios repentinos de humor, como pasar de estar contento(a) a estar triste o viceversa?	Sí_	No_
12.	¿Pierdes días de clase o llegas tarde a la escuela por haber consumido bebidas alcohólicas o drogas?	Sí_	No_
13.	¿Te han dicho tus familiares o amigos que debes reducir el uso de bebidas alcohólicas o drogas?	Sí_	No_
14.	¿Tienes discusiones serias con tus amigos o con miembros de tu familia por el uso que haces de bebidas alcohólicas o drogas?	Sí_	No_
15.	Cuando consumes bebidas alcohólicas o drogas, ¿tiendes a hacer cosas que normalmente no harías, tales como desobedecer reglas, violar leyes o llegar tarde a casa?	Sí_	No_
16.	¿Tienes dificultades en tus relaciones con alguno de tus amigos debido a las bebidas alcohólicas o drogas que consumes?	Sí_	No_
17.	¿Sientes que, a veces, no puedes controlar el deseo de consumir bebidas alcohólicas o drogas?	Sí_	No_

Cada respuesta afirmativa suma un punto. Se considera en riesgo de uso problemático, abuso o dependencia de alcohol o drogas un total de dos puntos o más.

ANNEX 2.

VERSIÓN ARGENTINA DEL CUESTIONARIO CRAAFT: CRAAFTa

PARTE A

Durante los últimos 12 meses:

1.	¿Has consumido bebidas alcohólicas (más de unos pocos sorbos)?	Sí_	No_
2.	¿Has fumado marihuana?	Sí_	No_
3.	¿Has usado algún otro tipo de sustancias que alteren tu estado de ánimo o de conciencia?*	Sí_	No_

* El término "algún otro tipo" se refiere a drogas ilícitas, medicamentos de venta libre o de venta con receta médica, así como a sustancias inhalables que alteren tu estado mental. (Si respondiste "SÍ" a CUALQUIERA de las anteriores 3 preguntas, pasa a las preguntas B1-B6).

PARTE B

1.	¿Alguna vez, viajaste en un vehículo conducido por vos u otra persona que hubiera consumido alcohol o drogas?	Sí_	No_
2.	¿Alguna vez, usaste alcohol o drogas para relajarte, sentirte mejor con vos mismo(a) o para integrarte a un grupo?	Sí_	No_
3.	¿Alguna vez, consumiste alcohol o drogas mientras estabas solo(a)?	Sí_	No_
4.	¿Alguna vez, te olvidaste de cosas que hiciste por haber consumido de alcohol o drogas?	Sí_	No_
5.	¿Alguna vez, tu familia o amigos te dijeron que disminuyeras el consumo de alcohol o drogas?	Sí_	No_
6.	¿Alguna vez, tuviste problemas por haber consumido alcohol o drogas?	Sí_	No_

Cada respuesta afirmativa suma un punto. Se considera en riesgo de uso problemático, abuso o dependencia de alcohol o drogas un total de dos puntos o más.

REFERENCES

1. Bauman A, Phongsavan P. Epidemiology of substance use in adolescence: prevalence, trends and policy implications. Drug Alcohol Depend. 1999; 55(3):187-207.         [ Links ]

2. Grant JD, Scherrer JF, Lynskey MT, Lyons MJ, et al. Adolescent alcohol use is a risk factor for adult alcohol and drug dependence: evidence from a twin design. Psychol Med. 2006; 36(1):109-18.         [ Links ]

3. Miech RA, Johnston LD, O'Malley PM, Bachman JG, et al. Monitoring the Future national survey results on drug use, 1975-2017: Volume I, Secondary school students [Internet]. Ann Arbor: Institute for Social Research, The University of Michigan, 2018. [Consulta: 21 de septiembre de 2018]. Disponible en: http://monitoringthefuture.org/pubs/monographs/mtf-vol1_2017.pdf.

4. ESPAD Group. ESPAD Report 2015: Results from the European School Survey Project on Alcohol and Other Drugs. Luxembourg: Publications Office of the European Union; 2016. [Consulta: 21 de septiembre de 2018]. Disponible en: http://www.espad.org/sites/espad.org/files/TD0116475ENN.pdf.

5. Observatorio Español de las Drogas y las Adicciones. Informe 2017. Alcohol, tabaco y drogas ilegales en España. Encuesta domiciliaria sobre alcohol y drogas en España (EDADES), 1995-2015. Madrid: Ministerio de Sanidad, Servicios Sociales e Igualdad; 2017. [Consulta: 21 de septiembre de 2018]. Disponible en: http://www.pnsd.mscbs.gob.es/profesionales/sistemasInformacion/sistemaInformacion/pdf/2017_Informe_EDADES.pdf.

6. Observatorio Español de las Drogas y las Adicciones. Encuesta sobre uso de drogas en enseñanzas secundarias en España (ESTUDES) 2016/2017. Madrid: Ministerio de Sanidad, Servicios Sociales e Igualdad. Secretaría de Estado de Servicios Sociales e Igualdad. Delegación del Gobierno para el Plan Nacional sobre Drogas; 2018. [Consulta: 21 de septiembre de 2018]. Disponible en: http://www.pnsd.mscbs.gob.es/profesionales/sistemasInformacion/sistemaInformacion/pdf/2016_2017_ESTUDES.pdf.

7. Secretaría de Programación para la Prevención de la Drogadicción y Lucha contra el Narcotráfico (SEDRONAR). Sexto estudio nacional sobre consumo de sustancias psicoactivas en estudiantes de enseñanza media 2014. Boletín N.° 1 Patrones y magnitud del consumo: diagnóstico a nivel país. [Consulta: 21 de septiembre de 2018]. Disponible en: http://observatorio.gob.ar/media/k2/attachments/AS_14337909681.pdf.

8. Wilson CR, Sherritt L, Gates E, Knight JR. Are clinical impressions of adolescent substance use accurate? Pediatrics. 2004; 114(5):e536-40.         [ Links ]

9. Knight JR, Sherritt L, Shrier LA, Harris SK, et al. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. 2002; 156(6):607-14.         [ Links ]

10. Bertini MC, Busaniche J, Baquero F, Eymann A, et al. Adaptación transcultural y validación del test CRAFFT como prueba de pesquisa para consumo problemático, abuso y dependencia de alcohol y otras sustancias en un grupo de adolescentes argentinos. Arch Argent Pediatr. 2015; 113(2):114-8.         [ Links ]

11. Rahdert ER. Problem Oriented Screening Instrument for Teenagers (POSIT). In: Rahdert ER (ed.). The Adolescent assessment/referral system manual. Rockville, MD: National Institute on Drug Abuse; 1991. p. 67-8.         [ Links ]

12. Knight JR, Goodman E, Pulerwitz T, DuRant RH. Reliability of the Problem Oriented Screening Instrument for Teenagers (POSIT) in adolescent medical practice. J Adolesc Health.2001; 29(2):125-30.         [ Links ]

13. Mariño MC, González Forteza C, Andrade P, MedinaMora ME. Validación de un cuestionario para detectar adolescentes con problemas por el uso de drogas. Salud Ment. 1998; 21(1):27-36.         [ Links ]

14. Organización Panamericana de la Salud y Consejo de Organizaciones Internacionales de las Ciencias Médica. Pautas éticas internacionales para la investigación relacionada con la salud con seres humanos. 4ta ed. Ginebra: Consejo de Organizaciones Internacionales de las Ciencias Médicas (CIOMS); 2016. [Consulta: 21 de septiembre de 2018]. Disponible en: https://cioms.ch/wp-content/uploads/2017/12/CIOMS-EthicalGuideline_SP_INTERIOR-FINAL.pdf.

15. Hair JF, Anderson RR, Tatham RL, Black WC. Multivariate Data Analysis. 4.a ed. Englewood Cliffs, NJ: Prentice Hall; 1995.         [ Links ]

16. Streiner DL. A checklist for evaluating the usefulness of rating scales. Can J Psychiatry. 1993; 38(2):140-8.         [ Links ]

17. Secretaría de Programación para la Prevención de la Drogadicción y Lucha contra el Narcotráfico (SEDRONAR). Sexto estudio nacional sobre consumo de sustancias psicoactivas en estudiantes de enseñanza media 2014. Boletín N.° 4. Patrones y magnitud del consumo: diagnóstico Región Centro. [Consulta: 21 de septiembre de 2018]. Disponible en: http://www.observatorio.gov.ar/media/k2/attachments/VIZEstudioZNacionalZaZEstudiantesZdeZNivelZMedioZCENTROZ-ZAoZ2014.pdf.

18. Reyna-Barajas GV, Copertari-Isaacson LF, González-Betanzos F, Padrós-Blázquez F. Estudio psicométrico del instrumento CARLOS (CRAFFT) en estudiantes universitarios mexicanos. Evaluar. 2016; 16(1):10-9.         [ Links ]

19. Rial A, Kim-Harris S, Knight JR, Araujo M, et al. Validación empírica del CRAFFT Abuse Screening Test en una muestra de adolescentes españoles. Adicciones. 2018:1105. [En prensa]. [Consulta: 21 de septiembre de 2018]. Disponible en: http://adicciones.es/index.php/adicciones/article/view/1105.

20. Wartberg L, Kriston L, Diestelkamp S, ArnaudN, et al. Psychometric properties of the German version of the CRAFFT. Addict Behav. 2016; 59:42-7. https://www.ncbi.nlm.nih.gov/pubmed/?term=Thomasius%20R%5BAuthor%5D&cauthor=true&cauthor_uid=27064750        [ Links ]

21. Subramaniam M, Cheok C, Verma S, Wong J, et al. Validity of a brief screening instrument-CRAFFT in a multiethnic Asian population. Addict Behav. 2010; 35(12):1102-4.         [ Links ]

22. Brown T. Confirmatory factor analysis for applied research. New York: The Guilford Press, 2006.         [ Links ]

23. Joreskog K, Sorbom D. PRELIS 2: User's reference guide. Chicago: Scientific Software Internacional, 1996.         [ Links ]

24. Dhalla S, Zumbo BD, Poole G. A review of the psychometric properties of the CRAFFT instrument: 1999-2010. Curr Drug Abuse Rev. 2011; 4(1):57-64.         [ Links ]

25. Winters KC, Stinchfield RD, Henly GA, Schwartz RH. Validity of adolescent self-report of alcohol and other drug involvement. Int J Addict. 1990-1991; 25(11A):1379-95.         [ Links ]

26. Gofin J, Gofin R. Atención primaria orientada a la comunidad: un modelo de salud pública en la atención primaria. Rev Panam Salud Pública. 2007; 21(2/3):177-84.         [ Links ]