ARTÍCULO ORIGINAL

Telementoring systems in the operating room: a new approach in medical training

 

Juan P. Wachs¹, Gerardo Gomez²

¹School of Industrial Engineering, Purdue University, Indiana, USA,
²Indiana University School of Medicine, Department of Surgery, Indianapolis, 46202, Indiana, USA

Postal address: J.P. Wachs, School of Industrial Engineering, Purdue University, West Lafayette, 47906, Indiana, USA
e-mail: jpwachs@purdue.edu

 

---

Abstract
This paper discusses the challenges and innovations related to the use of telementoring systems in the operating room. Most of the systems presented leverage on three types of interaction channels: audio, visual and physical. The audio channel enables the mentor to verbally instruct the trainee, and allows the trainee to ask questions. The visual channel is used to deliver annotations, alerts and other messages graphically to the trainee during the surgery. These visual representations are often displayed through a telestrator. The physical channel has been used in laparoscopic procedures by partially controlling the laparoscope through force-feedback. While in face to face instruction, the mentor produces gestures to convey certain aspects of the surgical instruction, there is not equivalent of this form of physical interaction between the mentor and trainee in open surgical procedures in telementoring systems. Even that the trend is to perform more minimally invasive surgery (MIS), trauma surgeries are still necessary, where initial resuscitation and stabilization of the patient in a timely manner is crucial. This paper presents a preliminary study conducted at the Indiana University Medical School and Purdue University, where initial lexicons of surgical instructive gestures (SIGs) were determined through systematic observation when mentor and trainee operate together. The paper concludes with potential ways to convey gestural information through surgical robots.

Key words: Telementoring; Surgical robotics; Gestures; Surgical instruction; Computer technology; Operating room

Resumen
"Telementoring" en el quirófano: un nuevo enfoque en la formación médica. Este artículo discute los desafíos e innovaciones relacionadas al uso de sistemas de tutoría en telecirugía (telementoring). La mayoría de los sistemas presentados se basan en tres tipos de canales de interacción: auditivo, visual y físico. El canal auditivo permite al instructor instruir verbalmente al alumno, y a este último hacer preguntas. El canal visual es usado para transmitir al alumno anotaciones, alertas y otro tipo de mensajes gráficos durante la cirugía. Estas representaciones visuales aparecen en un marcador de vídeo (telestrator). El canal físico ha sido usado en cirugías laparoscópicas por medio de retroalimentador de fuerza (forcefeedback). Mientras que en la instrucción cara a cara, el instructor hace gestos para transmitir ciertos aspectos de la instrucción quirúrgica, esta forma de interacción no tiene un equivalente en la interacción entre instructor y alumno en sistemas de telementoring. Si bien la tendencia es conducir procedimientos mínimamente invasivos (MIS) con estos sistemas, se deben tener en cuenta cirugías de trauma, todavía necesarias, especialmente donde la resucitación inicial y estabilización del paciente es un tema crítico y urgente. Este artículo presenta un estudio preliminar conducido en la Escuela de Medicina de Indiana (EE.UU.) y en la Universidad Purdue, donde el vocabulario de gestos (lexicons) usados en instrucción quirúrgica (SIGs) se determinaron por medio de observaciones sistemáticas mientras el instructor y el alumno operaban juntos. Se concluye discutiendo maneras alternativas de presentar esta información de gestos por medio de robots quirúrgicos.

Palabras clave: Telementoring; Cirugía robótica; Gestos; Instrucción quirúrgica; Tecnología informática; Quirófano

---

 

Telemedicine involves the use of information technologies and telecommunications to deliver healthcare to a remote location¹.Telementoring is an advanced form of telemedicine and telehealth, which is used to accomplish a dual role of educating trainee surgeons and delivering healthcare at the same time remotely². Over the last decade, telementoring has proven to give effective real-time guidance and instruction to trainees' surgeons in rural hospitals using audio, video, and haptics (nonverbal communication involving touch)³. In addition to a cost-effective, large-scale basis for clinical education oversight, it offers the availability of subspecialty surgical care in remote locations where this expertise may not be readily available⁴. Providing subspecialty expertise is necessary for underserved regions. Given that current clinical education emphasize specific sub-specialization areas in training, other areas receiving a lower priority or interest may never be learnt. That will leave the trainee with gaps in knowledge or lack of training in many subspecialty skills⁵. This is specially accentuated in rural areas, where the availability of surgeons with the appropriate surgical training expertise is lacking. Recently a teleconsultation and telementoring systems capable of addressing this problem were developed and validated in the clinical setting⁶. The main features of this robotic platform are a pan-and-tilt camera to deliver real-time imaging to the mentor, and a laser pointer attached to an overhead surgical light to allow the mentors indicate anatomical structures. The audio channel allows the mentor and mentee to communicate through speech. This platform was controlled by the subspecialist mentor located 30 miles away at the San Francisco Veteran Affairs Hospital.
Other telementoring systems leverage on additional communication channels, such as haptics, in addition to voice and visual communication. Haptics has been used in laparoscopic skills enhancement through mentoring⁷. A remote mentor guides the hands of a trainee surgeon as he/she executes complex procedures. As an example of this, is the robotic system developed by Nistor et al.⁸, where a high fidelity interface and a hierarchical control system generates resistances in the haptic device according to the surgical translated motions from both mentor and mentee inputs. Nevertheless, intervening physically during the surgery through surgical actions is still a virgin area of research.
A key objective of this paper is to discuss the feasibility of maximizing mentors' sense of presence by effectively increasing their physical participation through significant technological and scientific improvements.

Materials and Methods

Telementoring systems leverage on three types of interaction channels: audio, visual and physical. The audio channel enables two-way communications between the mentor and the trainee; allows the mentor to verbally instruct the trainee, and allows the trainee to ask questions. The visual channel is used to deliver feedback to the mentor about the patient anatomical layout, and the trainee dexterous performance. From the mentor side, it is possible to mark and annotate regions on the operative field of view and deliver those images to a monitor within the surgical site (e.g. telestrator). To indicate specific points directly on the patient's operative region, a laser pointer can be used to highlight those regions indicated remotely by the mentor. The physical channel has been used in laparoscopic procedures by partially controlling the laparoscope through haptics. Currently, there is not an equivalent for physical interaction between the mentor and trainee in open surgical procedures (e.g. endovascular, cardiac surgery and neurosurgery). While the trend is to perform more minimally invasive surgery (MIS), trauma surgeries are necessary, where initial resuscitation and stabilization of the patient in a timely manner is crucial.
On the mentor's front-end interface, visual feedback is used to deliver real-time imagery of the surgical work space, which includes the patient, the trainee, and some of the surgical instruments. These views are obtained by one or more cameras installed in the surgical site. This feedback is used for the mentor to construct a mental model of the surgical procedure, the trainee performance, the location of the instruments, and the patient's state. The state of the surgery, as visually perceived by the mentor, is used to give verbal directions to the trainee (if requested) or to intervene if deemed necessary; alternatively, the mentor can interact with the displayed data by annotating it (e.g. pointing, marking and selecting specific regions on the displayed imagery of the patient's anatomy). GUIs (graphical user interface), tablets, touch detection and gesture interaction can all be used to define the annotations.
The visual information delivered by the mentor is rendered to the trainee in either two ways: (a) projections on the patient's anatomy (for pointing), or (b) telestrator: a real-time image showing the mentor's annotations over the patient's anatomy as captured by the cameras. Both methods to provide feedback are limited and result in disrupting the trainee's immersion on the task. The laser pointer is currently being used just to point, while an in-situ mentor does much more than just pointing. Detailed studies^9-11 have shown how "surgeons coordinate their talk and delicate gestures with their hands and instruments when operating on a patient to create and configure a shared workplace and establish references to particular locations and features of the surgical field". While gesture has been seen as a supplement to verbal communication, the relationship is reciprocal. Current telementoring platforms impose certain restrictions on the mentor's ability to produce gestures. Embodiment⁹ is a critical factor that must be considered in the trainer-mentee relationship since they impart key non-verbal communication concepts which support the flow of information during surgery. Another problem is related to the use of telestrators: they require from the trainee to look away from the operative field, in order to view the mentor's annotations on the display. This causes distractions, delays or risks of injuries when holding sharps. A key research question is systematically determining those gestures that mentors perform during instructions, and how best implement them through telementoring. This paper will address the first question.
In order to build a lexicon of typical gestures used through surgery, in mentor-trainee relationships, a contextual validation was conducted through trauma, planned and training open surgeries at the Wishard Memorial Hospital (Indianapolis, IN). These procedures were observed initially and used to develop the initial lexicon of surgical instructive gestures (SIGs). The first trauma surgery consisted of repairing a vascular ischemic injury caused to male cyclist as a result of a traffic accident. The transected blood vessel in the leg was sutured and repaired by the vascular team and an angiogram was used to check proper intravascular flow. Afterwards, the fractured lower leg was aligned by the orthopedic team. This procedure required pairing the orthopedic surgeon with an orthopedic resident, which are part of the same surgical team. The surgeon used a small set of gestures to indicate the lesions' locations and the holding positions.
The second surgery required the repair of an overly dilated portion of the abdominal aorta (an open abdominal aortic aneurysm repair). This type of procedure involves the dissection and ligation of intervening veins, aneurysm resection, and repair, concluded by retroperitoneal and abdominal incisional wound closure. In this procedure, the number of SIGs used was larger than the previous procedure due to the complexity of the procedure, and its duration.
The third procedure observed was part of The Trauma Operative Management (ATOM) course which trains surgeons for damage-control laparotomies by using porcine model. Observations were recorded during February 2013, where a mentor surgeon is paired one to one with a resident to support complex procedures for improved mentorship. The scenario presented (through a porcine model) was that of 43-year old male stabbed in the lower abdomen which required repair of the intraperitoneal bladder laceration, and injury of the ileum. In this scenario, the SIGs were performed by the mentor surgeon, and often mirrored by the trainee. Mirroring seems to be a technique used by trainees to verify their understanding of the task to be conducted.
During the previous procedures, annotations, sketches (first two procedures), pictures and videos (last procedure) were taken by the researchers. The information gathered was post processed and reviewed in search of SIGs.
For the visual analysis, segments of the video footage were analyzed frame by frame, in search of meaningful gestures, and the supporting modality used to carry out those gestures (speech, proxemics, gaze, etc.). We identified the posture, trajectory and location of each of the mentor and trainee's hands, what they were doing, and what surgical instrument was held during the gesturing activity. For the analysis of the gestural component, the videos were broken down into clauses and reviewed for a range of grammatical and semantic signatures and patterns, which revealed the type of concept that the mentor was attempting to impart to the trainee. The set of all the gestures identified constitute the SIGs' lexicon used by the surgeons (See Fig.1).

Fig. 1.- SIGs' lexicon used by the surgeons (blue colored hands belong to the resident; green colored hands belong to the trainee): (a). Picking and Pointing; (b) Pointing with Instrument; (c) Pointing with Finger; (d) Pointing while Holding; (e) Holding; (f) Pulling and Exploring; (g) Palpating; (h) Requesting Tool

Results

Episodes including a mentor and trainee pair are presented here from three procedures. A SIG lexicon was obtained through the ATOM surgical trauma program, a challenging one to one mentoring training course involving the simulation of diverse blunt injuries on porcine models. During the procedure, the mentor (expert surgeon) must use gestures to indicate the places where incisions need to be performed, tissue need to be removed, or injuries need to be searched for. Furthermore, some gestures indicate how the organs need to be handled when dealing with significant bleeding. Overall these gestures are instructive gestures that well represent the tandem mentor-trainee surgical team.
Our observational and quantitative analysis shows: (i) the significance of non-verbal forms of communication, such as hand posture and pointing direction; and (ii) the relationship between the physical (nonverbal) communication form and what is being said and done in surgery. Here we present sketch diagrams obtained from real videos from the ATOM course (Fig. 1) and a legend describing the meaning of the gestures in the SIG lexicon. The mentor hands are green colored and those from the trainee are blue colored, to facilitate the visual analysis of the state of the training.
This episode begins with tissue incision on the left side of the porcine's abdomen performed by the resident in training, using an electrosurgical RF knife. The mentor points at the region of interest while picking and separating tissue (Fig. 1(a)). This gesture is referred as "Picking and Pointing", (PaP) since the mentor uses the forceps and his finger to point to the region of interest. Then, the mentor needs to engage both hands to keep the animal's vowels in place and at the same time point to the inflicted area with Mayo scissors. (Fig. 1(b)). This gesture is referred as "Pointing with Instrument", (PwI).
After that the mentor commands the resident to find the next region inflicted to the animal, while the resident presents a quick diagnosis of the animal. The mentor encourages the trainee to act by pointing emphatically with his finger. This is the "Pointing with Finger" (PwF) gesture ((Fig. 1(c)). At this point the trainee is taking the leading role, while suturing the inflected area. The mentor holds the side of the lineal alba of the porcine while pointing with the other hand. This gesture is called "Pointing while Holding" (PwH) and is presented in Fig. 1(d). In the next frame, the mentor takes a role of assistant to the trainee, by holding the suture being used. We refer to this type of expression as "Holding" (H) Fig. 1(e).
What occurs next is the mentor pulls the urethra of the animal, in order to enable the trainee to explore and find the reason of urine fluid in the abdominal cavity. These hand gestures will be called "Pulling and Exploring" (PaE) (see Fig. 1(f)). In the next still, Fig. 1(g) we see the mentor palpating the abdomen of the porcine after intraperitoneal packing (notice the swabs and towels). The gesture used is then referred as "Palpating" (P). In the last frame, the mentor again takes over the surgery, and requests surgical tools to finalize the surgical procedure. Therefore the last gesture is named "Requesting Tool" (RT), Fig. 1(h). In the last segment, the mentor and trainee end up switching sides (to allow the specialist to take over the procedure) and primary surgical roles are kept for the rest of the surgery.
Note that through the procedure it was noticed that the mentor and the trainee moved their heads and bodies, thus modulating in a way the explanations and inquires occurring during the procedure. Nevertheless, this study focuses on hand gestures since most of the instructional information was encapsulated through their movements and poses, and accompanied by spoken directions.
A total of eight gestures were found to be key components of the SIG's lexicon. While this set is not comprehensive by any means, it is still a faithful representation of the physical interaction through gestures occurring during mentorship and training in surgery.
Based on these findings, it is expected that the next phase in telementoring systems will allow the mentor to use instructional gestures through direct interaction with the patient and trainee using embodiment. Embodiment relates to the concept of enabling a robot to use its body as means of communication with humans. Thus, a future version of a telementoring system relying on surgical robots (see Fig. 2., Taurus robot, SRI International, Menlo Park, CA) should allow the demonstration and instruction of surgical tasks using gestures, in asimilar way that mentors and trainee use them in surgery when they are co-located.

Fig. 2.- The Taurus robot. Two robotic hands may enable both surgical tasks and gesture production.

Discussion

Several aspects of instruction during traditional mentoring in surgical education rely on the use of gestures for instructional purposes. Those gestures may involve one or two hands, and may occur while simultaneously performing or assisting during the surgical procedure. Standard telementoring technologies rely primarily on the transmission of verbal and visual information to the trainee located afar from the mentor. While this has been shown to present objective and subjective advantages to unimodal forms of interaction, physical expressions cannot be transmitted through this media. Our study postulates that embodied interaction can be used (through robotics) to support telementoring. This, in turn, will allow a more complete form of surgical training where both verbal and nonverbal instruction are conveyed remotely and at the same time efficiently.

Acknowledgements: The authors would like to thank Dr Brian Mullis for his consultancy and the use of the operating room (OR), Dr George Akingba for allowing observations at the IUSM teaching hospital's OR, and finally to Mrs. Shelly Marley to her insights related to surgical training and education.

Conflict of Interest: None to declare.

References

1. Perednia DA, Allen A. Telemedicine technology and clinical applications. JAMA 1995; 273: 483-8.         [ Links ]

2. Moore RG, Adams JB, Partin AW, Docimo SG, Kavoussi LR. Telementoring of laparoscopic procedures. Surgical endoscopy 1996; 10: 107-10.         [ Links ]

3. Chebbi B, Lazaroff D, Liu P. Collaborative virtual haptic environment for surgical training and tele-mentoring. Int J Robot Automat 2007; 22: 69-78.         [ Links ]

4. World Health Organization, Global observatory for e-health, series 2. Telemedicine: Opportunities and developments in Member States: Report on the second global survey on eHealth. Geneva, Switzerland: World Health Organization Press, 2010.         [ Links ]

5. Ereso AQ, Garcia P, Tseng E, et al. Live transference of surgical subspeciality skills using telerobotic proctoring to remote general surgeons. Journal of the American College of Surgeons 2010; 211: 400-11.         [ Links ]

6. Pande RU, Patel Y, Powers CJ, D'ancona G, Karamanoukian HL.The Telecommunication Revolution in the Medical Field: Present Applications and Future Perspective. Journal of Current Surgery 2003; 60: 636-40.         [ Links ]

7. Anvari M. Robot-assisted remote telepresence surgery. Seminars in laparoscopic surgery 2004; 11: 123-8.         [ Links ]

8. Nistor V, Carman GP, Faloutsos P, Allen B, Dutson E. Haptic Guidance for Laparoscopic Surgery Immersive Training and Mentoring. In: www.cse.yorku.ca/~pfal/Petros_Faloutsos/...files/NistorV-mmvr15.pdf        [ Links ]

9. Mondada L. Understanding as an embodied, situated and sequential achievement in interaction. Journal of Pragmatics 2011; 43: 542-52.         [ Links ]

10. Koschmann T, LeBaron C, Goodwin C, Feltovich P. Can you see the cystic artery yet? A simple matter of trust. Journal of Pragmatics 2011; 43: 521-41.         [ Links ]

11. Svensson MS, Heath C, Luff P. Instrumental action: the timely exchange of implements during surgical operations. In: ECSCW 2007;London: Springer; pp 41-60.         [ Links ]

Received: 4-X-2013
Accepted: 16-X-2013