SPECIAL ARTICLE

http://dx.doi.org/10.5546/aap.2017.eng.58

In-hospital capacity-building in research and management for pediatric professionals

 

Graciela Demirdjian, M.D.^a, Susana Rodríguez, M.D.^b, Juan C. Vassallo, M.D.^c, Vilma Irazola, M.D.^d and Josefa Rodríguez, M.D.^e

a. Coordinator, Health Technology Assessment, Hospital de Pediatría Juan P. Garrahan.
b. Associate Department of Teaching and Research.
c. Teaching Coordinator, Hospital de Pediatría Juan P. Garrahan.
d. Master of Clinical Effectiveness Program, Universidad de Buenos Aires.
e. Executive Medical Director, Hospital de Pediatría Juan P. Garrahan.

E-mail address: Graciela Demirdjian, M.D: gdemir@intramed.net

Funding: None.

Conflict of interest: None.

Received: 1-20-2016
Accepted: 9-07-2016

 

---

ABSTRACT

We describe an educational strategy aimed at capacity-building of hospital health care professionals in research and management initiated at a pediatric hospital in 2006, and the results obtained eight years after its implementation.

Research and Management in Pediatrics (GIP) is an annual 250-hour course combining meetings and off-site assignments delivered through the Hospital's on-line campus. It provides students with practical tools for research (epidemiology, methodology, bibliographic search, evidence-based medicine, biostatistics) and management (strategic planning, management programs, health services research, quality improvement, health economics). Assessment methods included integrative exercises, a final evaluation, and a group research or management project. Results obtained over the 2006-2013 period were highly satisfactory.

An intensive training program on research and management is a useful strategy for in-hospital capacity-building of pediatric health care professionals in basic tools for research activities, critical reading of biomedical literature and rational management ofpediatric health services.

Key words: Capacity-building; Hhospitals; Research; Management; Health economics.

---

 

INTRODUCTION

Evidence-based practice has already been recognized as a promoter of clinical effectiveness and efficiency in both medical education and health care management. There still persist, however, certain cultural and organizational barriers: resistance to change, limited human or technological resources, difficulties for accessing information, language barriers,¹ interdisciplinary consensus issues, lack of protected time for research or institutional support. Identifying and tackling those barriers may help health care organizations bridge the gap between evidence and practice, and make a positive impact on health care systems.²

There is evidence of the existence of enabling factors: tutoring, academic-clinical partnerships, institutional commitment,^3,4 development of critical thinking through participatory learning techniques (problem-based learning, bibliographic rounds, IT-supported interactive exercises, concept maps, discussion forums, debates, and simulation).^5,6

Health care professionals need to develop specific skills, such as asking precise questions and conducting efficient searches and critical assessments of biomedical literature, to ensure proper context-based decision-making. Few undergraduate disciplines place the focus on these competencies, and university students rarely anticipate the impact of these skills on their careers.

Capacity-building in research helps address care-related issues and generate new valid, applicable knowledge. Teaching of the scientific method in undergraduate and graduate education in different programs and universities is highly heterogeneous and extremely theoretical, and lacks practical applicability to research planning and implementation. These skills are acquired in an unsystematic fashion after frustrating experiences when replicating researches of poor scientific validity, at the time of communication at academic meetings or with failed publication attempts.

Management of professionals with leadership roles poses a similar challenge, as this requirement underpins quality and efficiency at different levels of the health care system,⁷ facilitates the implementation of scientific evidence in health care organizations or health policies, and

favors an impact assessment by means of usage studies or continuous monitoring of quality and cost indicators.⁸ Increasing demand from health services leaders seeking to improve their professional skills has encouraged a broad, varied offer of master and graduate specialization programs in hospital administration, health economics and quality improvement, the accessibility and applicability of which are often limited by time or financial constraints and overly theoretical approaches.

An intensive graduate training process, ideally on-the-job training that is tailored to the cultural and organizational context, is required to accomplish the goals set and bring about a paradigm shift. Brief courses and traditional passive education strategies disseminate information but fail to change behaviors.⁹ Attitude changes and the application of acquired skills require a multimodal approach employing active and dynamic techniques such as self-regulated learning, problem solving, interprofessional education and on-the-job mentoring and counseling.^10-12

This article describes the development and implementation methodology of an intensive program for capacity-building of hospital health care professionals in research and management initiated at a pediatric hospital, and the outcomes obtained eight years after its implementation.

PROGRAM DESCRIPTION

Research and Management in Pediatrics

(GIP) is an educational strategy designed and planned in 2004-2005 and initiated in April 2006 at Hospital de Pediatría "Prof. Dr. Juan P. Garrahan". Pediatric professionals from a wide array of disciplines (internists or specialists, pharmacists, biochemists, physical therapists, psychologists, nurses, and social workers) from the hospital were eligible for enrollment. Professionals from other public and private institutions have also been included since 2008.

Because the course is designed for intermediate-level training of in-house hospital staff, and since there are other courses available for pediatricians in training,¹³¹⁴ one requirement for admission was having earned the medical school diploma or having completed the basic residency program at least 5 years before course enrollment. Other nonexclusive requirements included being proficient at reading comprehension in English language and having sufficient free time to devote to study and off-site assignments. Minimum enrollment was established at 20 students and maximum enrollment, at 40 students annually.

This educational intervention seeks to provide pediatric professionals with practical tools for research and management activities. Students are expected to:

• Acquire methodological and statistical tools for planning, implementing and analyzing clinical research or health services projects.
• Apply technical evidence-based medicine knowledge to decision-making in clinical practice and health care administration.
• Assess the efficacy, safety, effectiveness and efficiency of therapeutic, preventive, organizational or educational health interventions.
• Incorporate practical skills for the administration and management of pediatric health services. GIP is an annual 250-hour intensive course featuring contents organized in two separate blocks (Table 1):

1. Research (epidemiology, methodology, bibliographic search and evidence-based medicine, biostatistics).
2. Management (strategic planning, management projects and programs, health services research, quality improvement, health economics).

Table 1. Theoretical contents of each block

Jointly organized by the Hospital and the faculty of the Master of Clinical Effectiveness Program of the Universidad de Buenos Aires, GIP is sponsored by the Universidad de Buenos Aires and supported by Fundación Garrahan. Permanent faculty (directors and coordinators, master program alumni) lead in-class activities and off-site assignments, and are available for guidance and counseling during tutoring hours. Teaching assistants are involved in planning and evaluation, and they participate as guest specialists on specific subjects and integration seminars open to other Hospital professionals and alumni for reflection and debate.

Contents are developed under the blended learning modality, with in-class, 3-hour, weekly lectures and workshops, and off-site assignments delivered in Moodle format through the Hospital's on-line campus. Students can log in to the campus to check schedules, download classes and required or supplementary bibliography, access suggested links to relevant websites, do interactive exercises, and participate in discussion forums.

The course is mostly practical: lectures account for only 30% of the course load; 10% is dedicated to practice workshops (web-based bibliographic search, critical reading, problem solving, use of STATA software for statistical analysis); and 60% the course load is devoted to study and instructional exercises (on-line self-assessment questionnaires with feedback, virtual interactive sequential application exercises) or evaluation exercises (students work in teams to solve problems related to block integration).

By mid-program, preliminary proposals for group research or management projects are discussed at a general meeting to optimize design and ensure feasibility. In the following months, the final project is developed under mentor supervision so that students can make any required adjustment and practice for their presentations. All groups participate in the final presentation. They are evaluated by a jury made up of 3 to 6 faculty members who use a checklist with pre-defined scores for assessing justification, goals, methodology, feasibility and impact, and presentation quality.

Final exams consist of a classroom component (multiple choice or short answer questions) and an off-site component (individual problem solving exercises related to research and management). Completing the instructional exercises and passing both the final exam and the group research or management project (accounting for 20%, 40% and 40% of final score, respectively) is mandatory for final certification.

By the end of the course, students complete a semi-structured satisfaction survey about course contents, methodology and applicability that is essential for making adjustments and changes.

The Moodle learning platform records data on the students' work and exercise results that can be downloaded as Excel spreadsheets for analysis and reporting. The following results were concluded from the analysis of such data and additional information gathered by the authors.

PROGRAM OUTCOMES

Between 2006 and 2013, 295 multidisciplinary pediatric professionals participated in the GIP program (general pediatricians or pediatric specialists: 75%; nurses: 8%; pharmacists: 5%; biochemists: 5%; physical therapists: 5%; nutritionists: 1%, and psychologists: 1%). Ninety-five percent (279) of participants were members of the hospital staff (25% belonged to the inhouse hospital staff) and 5% (16) were external participants. Minimum enrollment was met throughout all 8 course years, with an average of 35 enrollees (range: 25-43).

Seventy-seven group research (46%) and management (56%) projects were submitted dealing with a broad spectrum of subject matters: clinical research (46%), health services (23%), institutional programs (20%), quality improvement (6%), and economic assessments (5%). Many projects had an intra/extrahospital interdisciplinary or collaborative nature, and one third (32.5%) of them involved several members of the health team (different medical specialists, nurses, pharmacists, biochemists, physical therapists, nutritionists, psychologists). In the checklist evaluation, more than 50% of the 77 projects had a score of >80% (median: 8; range: 6-10 points).

Dropout was below 5%; 10% of participants completed the course assignments but decided not to be evaluated, and 6 students repeated the course before certification. Final certification was awarded to 85% (249) of course participants. The median grade for final exams was 87.5/100 (range: 61-99); only 4 students failed the final exam and decided to repeat the course (Figure 1). Global final grades ranged from 62 to 97/100, with a median of 85 (25^th percentile: 80; 75^th percentile: 88).

Figure 1. Exam grades

According to the satisfaction surveys, more than 90% of students were quite or very satisfied with course contents and assignments, job applicability and teacher availability for consultation and counseling. The main challenge was the lack of study time. Suggestions from students included increasing course hours, adding practical activities and offering short review and in-depth courses on specific subjects for program alumni.

DISCUSSION

It is essential for health care professionals to receive training in research and management so that they can successfully perform the multiplicity of tasks they are responsible for. Since training is approached from varied perspectives at different undergraduate programs, it must be included in graduate course syllabi. A positive impact on the health system is best accomplished if various relevant disciplines are included, with active participation and sufficient intensity to bring about behavior changes. Other short courses delivered at our institution (such as the Evidence-Based Medicine or Protocol Development workshops) were useful in raising awareness, but did not manage to make an impact in practice. GIP's comprehensive approach and the fact that it is an annual course allow for intensive training with less personal and economic effort than those required by graduate or specialization programs, which only few professionals can access.

Besides, its blended learning structure leverages two distinct approaches: the advantage of real-time teacher-student face-to-face contact of traditional learning and the enhanced participation and more efficient use of time afforded by virtual, asynchronous learning.¹⁵

On-site activities performed at work hours on rotating schedules each year favored the participation of hospital staff, as they were able to fit the course into their work routine. The on-line campus streamlined teacher-student communication¹⁶ and helped health teams become familiar with IT tools,^17,18 an allied benefit that is key for future evidence-based practice. IT support also allowed the faculty to track students' work, record and analyze data, and create reports.

Additional strategies led to enhanced participation and interdisciplinary interaction. Classes were interactive rather than lecture-structured, occasionally featuring games or humorous elements to help students release tensions and to contribute to a relaxed learning environment that favored active participation.^19,20 Group exercises and multidisciplinary projects helped create a broader outlook on clinical or organizational problems and more comprehensive solutions thanks to the collaborative effort between the members of the teams involved.²¹

Exams and self-assessment exercises were designed on the basis of real-life pediatric practice problems, and approached with varied techniques to reinforce everyday job applicability.^22,23 High grades in evaluations and projects indicate students achieved a satisfactory level of concept and tool mastery. Similarly suggestive are the results of satisfaction surveys and the low vacancy level, as annual enrollment was met without any course promotion effort: the program was mainly advertised by word of mouth.

Some published works agree with our observations about health care professionals' training needs, best methodologies for addressing them, and identified barriers.

In a questionnaire designed to probe the needs before the start of a research and management training program in the Pacific Islands, many physicians and nurses reported that they lacked basic research competencies (research protocol development, use of spreadsheets, critical reading of scientific papers).²⁴

There is a consensus that traditional short workshops in evidence-based medicine are insufficient to change behaviors, which calls for a comprehensive approach encompassing a multiplicity of levels and dimensions, including basic epidemiology concepts, knowledge transfer, guideline implementation and health policy development.^25,26

In line with this problem, the TEACH (Teaching Evidence Assimilation for Collaborative Health Care) program was developed and implemented in the United States, with a recent 5-year outcome report that highlights, as we do, the richness of multidisciplinary collaboration, process optimization using interactive activities and IT systems, on-the-job counseling to tackle context-specific problems, and the complexity of assessing the actual impact of these educational interventions.

A two-year program designed to build management capacity of public hospital leaders in Ethiopia is the published experience that best compares to ours; it underscores the need for comprehensive training (covering epidemiology, biostatistics, health economics, quality management) that is eminently practical (real-life problem solving, such as creating budgets) supported by IT systems, and with university collaboration.²⁷

Similar observations were reported in relation to a six-week community leadership development program in Cincinnati designed to enhance context-applied research capabilities in collaboration with academic partners. Many participants became involved in some type of activity in relation to acquired competencies (surveys, public database analysis, assessment of quality improvement programs or interventions, serving on advisory committees).²⁸

Hospital health care professionals' capacity building in research and management is one of the four pillars of the Health Technology Assessment Program implemented in the Hospital in 2001.²⁹ In 2014, the GIP course was redefined on the basis of outcomes, students' suggestions, and newly identified needs.

To meet the demand for more course hours and exercising, course duration was extended to 3 four-month terms, while maintaining the same course load by reducing in-class sessions from 3 to 2 hours a week. The Research block was divided into 4 modules (Protocol, Evidence-based Medicine, Bibliographic Search, and Biostatistics) distributed across the first 2 terms, and they can be taken consecutively or independently, which allows students to better organize themselves and favors the participation of alumni who need to review specific topics. The third four-month term (fifth module, Management) is delivered the following year, while the remaining portion of the year is devoted to project mentoring. With this restructuring, the complete course will be made available to new students every 2 years. It was already launched in 2015, with more than 40 regular students and alumni repeating individual modules.

Systematic data are not available yet for us to objectively quantify the long-term impact of this educational intervention on scientific output and hospital management.

Throughout the years, we have witnessed alumni engagement in research and management activities, such as research protocols, international publications, active serving on committees or working groups, development of multidisciplinary evidence-based guidelines, institutional programs, or proactive involvement in leadership of hospital services. With almost 300 hospital professionals having participated in this course (one fourth of the hospital's permanent staff), this critical mass is expected to bring about positive changes in the institution in the future, and objective measurement of this impact is a new challenge.

CONCLUSIONS

An intensive training program on research and management is a feasible, useful strategy for inhospital capacity-building of pediatric health care professionals in basic tools for research activities, critical interpretation of scientific evidence, and rational management of pediatric health services.

REFERENCES

1. Bernal G, Ortíz-TorresB. Barriers to Research and Capacity Building at Hispanic-Serving Institutions: The Case of HIV/AIDS Research at the University of Puerto Rico. Am J Public Health 2009;99(Suppl 1):S60-5.         [ Links ]

2. Newman M, Papadopoulos I, Sigsworth J. Barriers to evidence-based practice. Intensive Crit Care Nurs 1998;14(5):231-8.         [ Links ]

3. Fineout-Overholt E, Mazurek-Melnyk B, Schultz A. Transforming Health Care from the Inside Out: Advancing Evidence-Based Practice in the 21^st Century. J Prof Nurs 2005;21(6):335-44.         [ Links ]

4. Brady N, Lewin L. Evidence-Based Practice in Nursing: Bridging the Gap Between Research and Practice. J Pediatr Health Care 2007;21(1):53-6.         [ Links ]

5. Profetto-McGrath J. Critical Thinking and Evidence-Based Practice. J Prof Nurs 2005;21(6):364-71.         [ Links ]

6. Ridley RT. Interactive teaching: a concept analysis. J Nurs Educ 2007:46(5):203-9.         [ Links ]

7. Lannon CM, Levy FH, Moyer VA. The need to build capability and capacity in quality improvement and patient safety. Pediatrics 2015;135(6):e1371-3.         [ Links ]

8. Herrin D, Jones K, Krepper R, Sherman R, Reineck C. Future Nursing Administration Graduate Curricula. Part 2: Foundation and Strategies. J Nurs Adm 2006;36(11):498-505.         [ Links ]

9. Grimshaw JM, Eccles MP, Walker AE. Changing physicians' behavior: What works and thoughts on getting more things to work. J Contin Educ Health Prof2002;22(4):237-43.         [ Links ]

10. Ciliska D. Educating for Evidence-Based Practice. J Prof Nurs 2005;21(6):345-50.         [ Links ]

11. Lyon AR, Stirman SW, Kerns SE, Bruns EJ. Developing the Mental Health Workforce: Review and Application of Training Approaches from Multiple Disciplines. Adm Policy Ment Health 2011;38(4):238-53.         [ Links ]

12. Danielson J, Craddick K, Eccles D, Kwasnik A, et al. A Qualitative Analysis of Common Concerns about Challenges Facing Pharmacy Experiential Education Programs. Am J Pharm Educ 2015;79(1):6.         [ Links ]

13. Demirdjian G, Converso G, Echezarreta E, Rivarola M, Belgorosky A. Evaluación de una estrategia diferente para la enseñanza de la Metodología Científica en un hospital pediátrico. Med Infant 2001;8(3):210-7.         [ Links ]

14. Demirdjian G. Evaluación de la realización de talleres de diseño de investigación en el postgrado. Med Infant 1996;3(2):95-8.         [ Links ]

15. Howatson-Jones L. Designing web-based education courses for nurses. Nurs Stand 2004;19(11):41-4.         [ Links ]

16. Freire LL, Arezes PM, Campos JC. A literature review about usability evaluation methods for e-learning platforms. Work 2012;41(Suppl 1):1038-44.         [ Links ]

17. Barnard A, Nash R, O'Brien M. Information Literacy: Developing Lifelong Skills Through Nursing Education. J Nurs Educ 2005;44(11):505-10.         [ Links ]

18. Childs S, Blenkinsopp E, Hall A, Walton G. Effective e-learning for health professionals and students - barriers and their solutions. A systematic review of the literature - findings from the HeXL project. Health Info Libr J 2005;22(Suppl 2):20-32.         [ Links ]

19. Baid H, Lambert N. Enjoyable learning: The role of humour, games, and fun activities in nursing and midwifery education. Nurse EducToday 2010;30(6):548-52.         [ Links ]

20. Chauvet S, Hofmeyer A. Humor as a facilitative style in problem-based learning environments for nursing students. Nurse Educ Today 2007;27(4):286-92.         [ Links ]

21. Posey L, Pintz C. Online teaching strategies to improve collaboration among nursing students. Nurse Educ Today 2006;26(6):680-7.         [ Links ]

22. Epstein R. Assessment in Medical Education. N Engl J Med 2007;356(4):387-96.         [ Links ]

23. Schilling K, Applegate R. Best methods for evaluating educational impact: a comparison of the efficacy of commonly used measures of library instruction. J Med Libr Assoc 2012;100(4):258-69.         [ Links ]

24. Ekeroma AJ, Kenealy T, Shulruf B, McCowan LM, et al. Building reproductive health research and audit capacity and activity in the pacific islands (BRRACAP) study: methods, rationale and baseline results. BMC Med Educ 2014;14:121-30.         [ Links ]

25. Eddy DM. Evidence-Based Medicine: A Unified Approach. Health Aff (Millwood) 2005;24(1):9-17.         [ Links ]

26. Wahabi HA, Al-Ansary LA. Innovative teaching methods for capacity building in knowledge translation. BMC Med Educ 2011;11:85-94.         [ Links ]

27. Kebede S, Abebe Y, Wolde M, Bekele B, et al. Educating leaders in hospital management: a new model in SubSaharan Africa. Int J Qual Health Care 2010;22(1):39-43.         [ Links ]

28. Crosby LE, Parr W, Smith T, Mitchell MJ. The Community Leaders Institute: An Innovative Program to Train Community Leaders in Health Research. Acad Med 2013;88(3):335-42.         [ Links ]

29. Demirdjian G. A 10-year hospital-based health technology assessment program in a public hospital in Argentina. Int J Technol Assess Health Care 2015;31(1-2):103-10.         [ Links ]