ORIGINAL ARTICLE

http://dx.doi.org/10.5546/aap.2018.eng7

ASQ-3: Validation of the Ages and Stages Questionnaire for the detection of neurodevelopmental disorders in Argentine children

 

Ana M. Romero Otalvaro, M.D.^a-^b-^c, Nora Grañana, M.D.^a,b, Nadia Gaeto, M.D.^a, María de los A. Torres, B.S.^a, María N. Zamblera, B.S.^a, María A. Vasconez, B.S.^a,d, Claudia Misenta, M.D.^a, María E. Rouvier, M.D.^a and Jane Squires, M.D.^e

a. Department of Pediatrics, Division of Mother and Child Health, Hospital Carlos Durand, Autonomous City of Buenos Aires.
b. Universidad Maimónides, Buenos Aires.
c. Universidad Pontificia Bolivariana. Montería, Colombia.
d. School of Psychology, Secretariat for Higher Education, Science, Technology and Innovation (Secretaría de Educación Superior, Ciencia, Tecnología e Innovación, SENESCYT), Ecuador.
e. Department of Special Education and Clinical Sciences, School of Education, University of Oregon, Oregon, United States of America.

E-mail address: Nora Grañana, M.D.: ngranana@hotmail.com

Funding: None.

Conflict of interest: None.

Received: 12-9-2016
Accepted: 7-10-2017

 

---

ABSTRACT

Introduction. The systematic assessment of child development in the first years of life is an essential component of pediatric health checkups. The Ages and Stages Questionnaire, third edition (ASQ-3) is the most validated scale, and has been recommended by the UNICEF to verify if children have a normal neurological development. It is a monitoring instrument to assess the main developmental areas, including communication, gross motor, fine motor, personal-social, and problem solving skills, and to compare the local population to the international development standards.
Objective. To validate the ASQ-3 in a pediatric population group.
Methods. Children aged 1-66 months were assessed at a public hospital by pediatricians, psychologists, and educational psychologists. The SSPS software package was used to determine population scales. Results. In 630 children, who had a homogeneous sex distribution, an 88% sensibility and a 94% specificity were determined, with a positive predictive value of 88% and a negative predictive value of 96%, compared to the National Screening Test (Prueba Nacional de Pesquisa, PRUNAPE) and the cut-off scores for each age group. Conclusion. The ASQ-3 established that 19.5% of children were at risk of experiencing neurodevelopmental disorders. The ASQ-3 met psychometric properties compared to the PRUNAPE, which is the gold standard for the targeted and systematic assessment of developmental milestones during health checkups in a rapid, simple and cost-effective manner, so it was considered useful to monitor child neurological development.

Key words: Child; ASQ-3; Surveys and questionnaires; Detection; Neurodevelopmental disorders.

---

 

INTRODUCTION

The United Nations International Children's Emergency Fund (UNICEF) promotes the use of structured scales to detect neurodevelopmental disorders.¹ The World Health Organization (WHO) considers that at least 5% of the population suffers a psychomotor development disorder,² which is more common in developing countries.^3,4

The Committee on Children of the American Academy of Pediatrics (AAP) recommends that preventive care visits throughout the first 5 years of life should include a structured and systematic surveillance of development using standardized developmental screening tests administered, at least, at three ages (9, 18, and 24 or 30 months old) and autism detection tests should also be administered at 18 and 36 months old.⁵ This is based on different studies that have demonstrated that pediatricians' clinical judgment is not enough to identify delays during health checkups because they are not able to detect 30-50% of psychomotor development deficits so that early interventions could be implemented.^6-8

Detection may be done using questionnaires or tests, depending on their administration (Table 1). The Denver Developmental Screening Test II (DDST-II)⁹ helps to detect developmental problems occurring between 0 and 6 years old. It explores four areas: gross motor, fine motoradaptive, language, and personal-social skills, which are assessed by observing the child's performance and asking questions to parents. Performance is classified into normal, suspect or developmental delay. It is an extensively used American test with a high specificity level but a low to moderate sensitivity.⁵ In Latin America, the National Screening Test (Prueba Nacional de Pesquisa, PRUNAPE) is the adaptation of the DDST-II and has been validated in a population of 106 children in Argentina.¹⁰ Its biggest advantages are that it was adapted based on typical cultural traditions and was a ground-breaking tool that has been used in Latin America. In relation to its disadvantages, besides the lack of sensitivity of the DDST-II, it requires specific professional training, therefore making it expensive and taking time of the staff from socioenvironmental risk areas away from training centers. For this reason, the PRE-PRUNAPE questionnaire was introduced, which can be administered in a simpler manner but has a low sensitivity, approximately 43%.¹¹

Table 1. Description of child neurological development screening tools

Local early diagnosis test.

Squires increased detection by attaining a 90% sensitivity and a 91% specificity with the validation of the 1999 Ages and Stages Questionnaire (ASQ),¹² the ASQ third edition from 2009,¹³ and the ASQ:SE (social emotional).¹⁴ It may be self-administered or the questions may be asked by a pediatrician or any other health care provider or education professional, it does not require specific training, and is rapidly administered. It saves time during the pediatric office visit and, in more than 80% of cases, families are very interested in completing a structured follow-up of their children's development.⁶

In 2002, the AAP recommended the use of a systematic screening and this resulted in a more than two-fold increase of their use by pediatricians during health checkups¹⁵ and of treatment access for 80-90% of screened children. At least 86.7% of the Argentine urban population undergoes health checkups throughout the first 3 years of life; of these, more than a half (51.6%) are done in the public health sector, especially primary health care centers, facilitating the administration of this structured questionnaire.

OBJECTIVES

• The primary objective of this study was to assess the ASQ-3's ability to determine whether a child is at risk for developmental delay.

• As a secondary objective, the ASQ-3's psychometric properties were established and the cut-off scores from 1 to 66 months old were determined.

• Lastly, the frequency of neurodevelopmental disorders in the studied population was assessed.

METHODS

A quantitative, non-experimental, transactional, and descriptive study was done with non-probability sampling, where the collected sample had the same proportion of subjects than the entire population in relation to the studied phenomenon.

A total of 2780 children aged 1-66 months attended the Teaching office for a health checkup at Hospital C. Durand and the hospital's daycare center between March 2013 and February 2014. Of them, 60% came from the central-western area of the Autonomous City of Buenos Aires and approximately 40%, from the first and second belts of Greater Buenos Aires. All participants signed an informed consent and the study was approved by the Ethics Committee. A quota sample of 30 subjects was obtained at random from each of the 21 age groups resulting from the original questionnaire to include the variability range of the child population, collected consecutively.

The exclusion criteria were as follows: as a reason for consultation, children with developmental disorder, high perinatal risk, diagnosis of auditory or visual sensory disorder, and clinical or neurological chronic conditions at risk for developmental delay.

The Graffar Méndez¹⁶ demographic survey was used to determine the family socioenvironmental level, and the ASQ-3, in all cases compared to the PRUNAPE,¹⁰ was used as the gold standard to establish the test's validity.

The Graffar Méndez¹⁶ survey is based on the study of family social characteristics, employment of the head of the household, level of maternal education, sources of family income, and household comfort. Social status was established based on the sum of these scores; families living in relative poverty and extreme or critical poverty conditions have the highest scores (IV and V). Social strata were correlated to government and UNICEF statistics.¹⁷

The ASQ-3 is a child developmental progress screening questionnaire including 30 questions from 5 domains: communication, gross motor, fine motor, problem solving, and personal-social, based on milestones that should be achieved between 0 months old and 5.5 years old.¹³ It may be completed by a non-specialist observer or selfreported by parents, and has a 93% reliability. The characteristics described in Table 1 show that the ASQ-3 is the most validated scale in the developmental surveillance field¹⁸ and has been recommended by the UNICEF in the assessment toolkit for emerging countries.¹

It was decided to have the parent questionnaire completed by the health care provider to prevent any comprehension difficulties resulting from a lower level of reading skills. "Yes" was marked to indicate that their child demonstrated the ability to do the specific activity described by the item, which corresponded to 10 points. "Sometimes" indicated that the skill was emerging and corresponded to 5 points, and "Not yet" was selected to indicate that the child had not yet shown evidence of the ability to do that specific activity, and awarded 0 points. The sum of all items in each domain is transferred to a grid to classify performance based on the cut-off points established for each age and domain (Annex). The sum of each questionnaire by cut-off level may fall into one of three areas:

• White area: the child is developing according to expectations.

• Gray area: the child is developing in the borderline of expectations; it corresponds to a score < 1 standard deviation from the mean (-1 SD). Stimulation guidelines may be provided and rescreening should be scheduled one month later.

• Black area: the child's performance is below expectations and he/she shows difficulties and requires a referral for a diagnostic evaluation by an appropriate provider; it corresponds to a score < -2 SD of the mean.

The screening is considered positive if the child's score falls below expectations in at least one domain.

Mean and SD for each month and domain were estimated based on a non-parametric analysis, which did not require the normality of outcome measures. The calculated percentiles established the cut-off points to classify the sample into normal, moderately atypical, and extreme groups, described in the grids by age.

The ASQ-3 was compared to the PRUNAPE administered by the same experimented provider on a different day. The pRuNAPE is a set of standardized developmental items for healthy children aged 0-5.99 years including question and test items for each age group.¹⁰ A population made up of 106 children from Argentina with adequate sensitivity and specificity psychometric properties was established and is detailed in Table 1.

The test's validity was determined using sensitivity, specificity, positive predictive value, and negative predictive value parameters. A descriptive analysis of the studied outcome measures was done using a box plot; then normality tests were done for each domain in each age group using two non-parametric tests: the Shapiro-Wilk normality test and the Kolmogorov-Smirnov test; lastly, and considering the nature of distribution in each subgroup (domain and age), the Z-scores were estimated as 0, -1, and -2 SD. This way, the cut-off points were identified to establish a classification into high, middle and low. According to results, the distribution was abnormal (Figure 1), so non-parametric tests were used for data processing.

Figure 1. Descriptive analysis of development outcome measures

The SSPS 15 package was used for statistical processing. The age groups into which the test was segmentedComo usar galochaswere established as independent outcome measures and the ASQ-3 domains, as dependent outcome measures. The cut-off points were established by identifying sensitivity, specificity, positive predictive value, and negative predictive value.

Children who showed neurodevelopmental deficit in the screening were referred for neurological and neuropsychological assessment to complete the diagnosis and start a treatment.

RESULTS

A total of 630 children aged 1-66 months and with a homogeneous sex distribution were assessed; they had been randomly selected from the hospital's health checkups and daycare center and whose parents had agreed to participate. Thirty-five children were excluded because they had attended to consult about a delay in developmental milestone achievement. The overall sample was used to make a local adaptation and establish scales.

It was determined that the questionnaire administered to the mothers took 10-12 minutes among 95% of the sample. It was administered in a short period of time and easily scored. It was also easy to train the rest of the hospital staff, and it only required a scoring sheet, pencil and paper, and a sheet with stimulation guidelines to be delivered to the parents who took the questionnaire, which were the only expenses besides the initial kit, so it was considered a cost-effective tool in our country to control healthy children. In relation to the social composition of the final sample, socio-demographic strata were described using the Graffar Méndez survey. Most subjects corresponded to a middle socio-economic level III (20%), and IV (27%), and to a low level V (33%). Socio-economic strata distribution corresponding to high socio-economic levels I (7%) and II (13%) was in accordance with the local population.

Mean and SD values by age are described in the Annex. The ASQ-3's empirical validity was acceptable, both in terms of sensitivity (88%) and specificity (94%), and in relation to positive predictive value (88%) and negative predictive value (96%) compared to the PRUNAPE. A contingency table was developed to estimate the agreement level between the ASQ-3 and the PRUNAPE (Table 2). Results show a yf test value of y2 = 300.554 and a P value < 0.01; therefore, there is evidence of a highly significant level of dependence among results. The testing correlation for the overall analysis to identify the deficit and normal subjects was significant (r: 0.81; p: 0.00).

Table 2. Descriptive statistics and agreement between the Ages and Stages Questionnaire and the National Screening Test

As a result, 19.5% of children were categorized as being at risk and 12.6% as having a clinical diagnosis in one or more domains. Tables 3 and 4 describe in detail the frequencies by age and developmental area.

Table 3. Frequency of neurodevelopmental deficit by age group according to the Ages and Stages Questionnaire-3 in 123 children

Table 4. Frequency of risk for deficit by developmental area in 123 children

DISCUSSION

The primary objective of this study was to assess the ASQ-3's ability to determine whether a child is at risk for developmental delay. The most important bias observed when extrapolating the ASQ to other countries is language. In 2009, the questionnaire was validated in different countries, such as Chile¹⁹ and Peru,²⁰ using the Spanish version developed by Ellen McQuilkin.

The answers given by parents about their children are reliable²¹ because inter-observer reliability studies reported that the assessment made by health care providers is consistent with that made by parents, especially if the latter answer questions about their children's present development.¹² Parental observations, even considering the differences by socio-economic level, geographic location or parental well-being, provide reliable information about their children's development,^22,23 with a testing-retesting reliability after two weeks above 90%.¹²

The original ASQ-3 study assessed 15 138 North American children. The psychometric properties were established with a high specificity, sensitivity, positive predictive value, and negative predictive value. It showed a high specificity and negative predictive value; for a screening scale, this meant that if the ASQ-3 had a normal result, it was highly unlikely for a child to have a developmental deficit, i.e., it minimized the possibility of failing to detect children with an actual developmental delay, one of the main strengths of the questionnaire.²⁴ These psychometric properties were maintained for the local adaptation.

Cut-off scores were determined for 1 to 66 months old. Compared to the North American standards, the Buenos Aires population showed a lower performance, especially in the communication and problem solving areas as of 12 months old.¹³

It is important to consider the population cutoff scores for comparison purposes. The studies conducted by Rubio-Codina et al. in Colombia did not find a correlation between performance in the ASQ-3 and the Bayley Scales of Infant and Toddler Development -American versions-without considering population characteristics and local validations for any of the two scales.²⁵ A pilot study to compare the ASQ-3, the PRUNAPE, and the PRE-PRUNAPE screenings in Argentina showed that the ASQ-3 had the highest specificity and was more easily implemented.²⁶

Lastly, the frequency of neurodevelopmental disorders in the studied population was assessed, and it was consistent with the frequency of developmental delay reported for South American countries, with a higher incidence of deleterious socio-environmental factors (violence, inattention or neglect), growth deficit, and disease control.¹⁹ The predominance of low social levels (IV and V) was consistent with the local population epidemiology.¹⁷ These results were lower, especially as of 12 months old, which was then maintained at all ages, similarly to what was observed in the validation for the Colombian population,²⁷ compared to the original sample from the United States.

The biggest limitation of this study was the use of a local population. It could be extended to other Argentine regions and larger population samples so as to obtain homogeneous standards and greater diversity.

For future guidelines, it would be interesting to reproduce and extend results to different populations and to establish their broad, systematic use among pediatricians during health checkups. Multivariate samples should be used to establish how the questionnaire functions in children with special needs. In addition, its use could be studied in populations with risk scores to measure the effectiveness of early interventions among children with scores falling in the gray area.

CONCLUSIONS

• In the studied population, the ASQ-3 effectively discerned in a short period of time which children were suspected to have a neurodevelopmental problem.

• It was observed that the ASQ-3 met the psychometric properties necessary for a targeted and systematic assessment of development during health checkups with cut-off scores adapted to the local 1-66-month-old population compared to the gold standard screening test, the PRUNAPE.

• Also, 19.3% of children were categorized as having a deficit based on the test standards in the studied population.

The ASQ-3 was valid as a standardized method for the assessment of neurological development in the studied population.

ANNEX
Answer grids for the Argentine population based on the established cut-off points

REFERENCES

3. Halpern R, Giugliani ERJ, Victora CG, et al. Fatores de risco para suspeita de atraso no desenvolvimento neuropsicomotor aos 12 meses de vida. J Pediatr (Rio J) 2000;76(6):421-8.         [ Links ]

4. Figueiras A, Souza I, Ríos V, et al. Manual para la vigilancia del desarrollo infantil (0-6 años) en el contexto AIEP. Washington: Organización Panamericana de la Salud, 2006.         [ Links ]

5. Council on Children With Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; et al. Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics 2006;118(1):405-20.         [ Links ]

6. Rydz D, Srour M, Oskoui M, et al. Screening for developmental delay in the setting of a community pediatric clinic: A prospective assessment of parent-report questionnaires. Pediatrics 2006;118(4):e1178-86.         [ Links ]

7. Mackrides PS, Ryherd SJ. Screening for developmental delay. Am Fam Physician 2011;84(5):544-9.         [ Links ]

8. King T, Tandon S, Macias M, et al. Implementing developmental screening and referrals: lessons learned from a National Project. Pediatrics 2010;125(4):350-60.         [ Links ]

9. Frankenburg WK, Goldstein AD, Camp BW. The revised Denver Developmental Screening Test: its accuracy as a screening instrument. J Pediatr 1971;79(6):988-95.         [ Links ]

10. Lejarraga H, Kelmansky D, Pascucci MC, et al. Prueba Nacional de Pesquisa PRUNAPE manual técnico. Buenos Aires: Fundación Hospital Garrahan, 2005.         [ Links ]

11. Lejarraga H, Kelmansky D, Lejarraga C, et al. Validación de un formulario para la detección de niños con alto riesgo de padecer trastornos del desarrollo: Cuestionario PRUNAPE pre-pesquisa. Arch Argent Pediatr 2013;111(6):476-83.         [ Links ]

12. Squires J, Bricker D, Potter L. Revision of a parent-completed developmental screening tool: Ages and Stages Questionnaires. J Pediatr Psychol 1997;22(3):313-28.         [ Links ]

13. Squires J, Bricker D. Ages & Stages Questionnaires (ASQ-3): A Parent-Completed Child-Monitoring System. 3rd ed. Stanford: Paul Brookes Publishing Company, 2009.

14. Squires J, Bricker D, Twombly E. ASQ-SE-2 User's Guide. 2nd ed. Baltimore: Paul Brookes Publishing Company, 2015.         [ Links ]

15. Radecki L, Sand-Loud N, O'Connor KG, et al. Trends in the use of standardized tools for developmental screening in early childhood: 2002-2009. Pediatrics 2011;128(1):14-9.         [ Links ]

16. Méndez Castellano H, Méndez MC. Sociedad y Estratificación: Método Graffar Méndez Castellano. Caracas: Fundacredesa, 1994.         [ Links ]

17. Ministerio de Desarrollo Social de la Nación, UNICEF. Encuesta sobre condiciones de vida de niñez y adolescencia. Principales resultados. 2011-2012. [Acceso: 13 de julio de 2017]. Disponible en: http://www.unicef.org/argentina/spanish/MICS_CierreJulio2013_BAJAWEB.pdf.

18. Macy M. The evidence behind developmental screening instruments. Infants Young Children 2012;25(1):19-61.         [ Links ]

19. Schonhaut L, Armijo I, Schonstedt M, et al. Validity of the ages and stages questionnaires in term and preterm infants. Pediatrics 2013;131(5):e1468-74.         [ Links ]

20. Kyerematen V, Hamb A, Oberhelman RA, et al. Exploratory application of the Ages and Stages (ASQ) child development screening test in a low-income Peruvian shantytown population. BMJ open 2014;4(1):e004132.         [ Links ]

21. Sheehan R. Involvement of parents in early childhood assessment. In: Wachs T, Sheehan R, editors. Assessment of young developmentally disabled children. New York: Plenum Press; 1988.Págs.75-90.         [ Links ]

22. Glascoe F. Parent's evaluation of developmental status: how well do parent's concerns identify children with behavioral and emotional problems? Clin Pediatrics (Phila) 2003;42(2):133-8.         [ Links ]

23. Gardner W, Kelleher K, Pajer K, et al. Primary care clinician's use of standardized tools to assess child psychosocial problems. Ambul Pediatr 2003;3(4):191-5.         [ Links ]

24. Meisels S, Atkins-Burnett S. Developmental screening in early childhood: A guide. 5th ed. Washington, DC. National Association for the Education of Young Children; 2005.         [ Links ]

25. Rubio-Codina M, Araujo MC, Attanasio O, et al. Concurrent validity and feasibility of short tests currently used to measure early childhood development in large scale studies. PloS One 2016;11(8):e0160962.         [ Links ]

26. Stewart C. Developmental Assessment in Argentina: A Comparison of Two Screening Instruments [Tesis doctoral]. Eugene, OR: University of Oregon; 2014. [Acceso: 13 de julio de 2017]. Disponible en: https://scholarsbank.uoregon. edu/xmlui/handle/1794/18466.

27. Romero Otalvaro AM. Validación y adaptación de Ages and Stages Questionnarie 3 Ed en Colombia. [Tesis doctoral]. Buenos Aires: Universidad Maimónides; 2015.

28. Rodríguez S, Arancibia V, Undurraga C. Escala de Evaluación del Desarrollo Psicomotor de 0 a 24 meses. Santiago de Chile: Galdoc; 2001:18-54.         [ Links ]