Atormac
briv
Neurology India
menu-bar5 Open access journal indexed with Index Medicus
  Users online: 3085  
 Home | Login 
About Editorial board Articlesmenu-bullet NSI Publicationsmenu-bullet Search Instructions Online Submission Subscribe Videos Etcetera Contact
  Navigate Here 
 Search
 
  
 Resource Links
  »  Similar in PUBMED
 »Related articles
  »  Article in PDF (1,419 KB)
  »  Citation Manager
  »  Access Statistics
  »  Reader Comments
  »  Email Alert *
  »  Add to My List *
* Registration required (free)  

 
  In this Article
 »  Abstract
 »  Materials and Me...
 » Results
 » Discussion
 » Conclusion
 »  References
 »  Article Figures
 »  Article Tables

 Article Access Statistics
    Viewed1008    
    Printed2    
    Emailed0    
    PDF Downloaded22    
    Comments [Add]    

Recommend this journal

 


 
Table of Contents    
BRIEF REPORT
Year : 2021  |  Volume : 69  |  Issue : 3  |  Page : 698-702

Feasibility of Conducting a Virtual Exit Exam in Neurosurgery During the SARS-COV19 Pandemic


1 Department of Neurosurgery, PGIMER, Chandigarh, India
2 Department of Neurosurgery, CMC Vellore, Tamilnadu, India
3 Department of Neurosurgery, Guwahati Medical College, Guwahati, Assam, India

Date of Submission02-Nov-2020
Date of Decision27-Jan-2021
Date of Acceptance31-Mar-2021
Date of Web Publication24-Jun-2021

Correspondence Address:
Dr. Sushanta Kumar Sahoo
Assistant Professor, Neurosurgery, Room No -27, 5th Floor, F Block, Nehru Hospital, PGIMER, Chandigarh, Assam 160012
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0028-3886.319207

Rights and Permissions

 » Abstract 


Background: Skills assessment forms an integral part of the exit examination in neurosurgical training programs. The established method of evaluating trainees for their clinical knowledge and surgical proficiency in the operating room is not feasible in the current time of SARS-COV19 pandemic.
Objective: The feasibility of conducting such an assessment using case modules on an online meeting platform is discussed.
Methods: Six candidates were evaluated on two consecutive days with two internal examiners located at the examination site and two external examiners situated at their own institutions elsewhere in the country. Clinical details, including images and videos of patients managed at our institute were recorded and provided to the candidates as case modules. Four sessions were conducted in the form of long and short cases, operative neurosurgery, neuroradiology, and neuropathology and a general viva-voce using “Zoom” (Zoom-Video-Communications, Inc. USA) platform. Feedback from the examinee and the examiners were obtained for any modification in the current format.
Result: The online platform worked well without any interruption except for slight lag in the audio-visual system and occasional difficulty in using microphone and screen simultaneously. Trainees were able to interpret the clinical details and rated this format close to actual clinical evaluation. The examiners uniformly agreed that the online format for assessment was satisfactory and made some suggestions for improvement.
Conclusion: Clinical and surgical skill evaluation is feasible using case modules and online meeting platforms. Use of original patient's data, images, videos demonstrating clinical signs, and operative procedures makes this assessment more objective.


Keywords: SARS-COV19 pandemic, exit exam, neurosurgery
Key Message: Traditional neurosurgical exit exams are not feasible during this ongoing COVID 19 pandemic. Online meeting platforms with good audio-visual facility along with case scenarios, operative case modules, and radiology in power point format can be used for conducting such assessments.


How to cite this article:
Salunke P, Sahoo SK, Chacko AG, Baishya BK, Tripathi M, Chabbra R, Karthigeyan M, Aggarwal A, Singh A, Gupta SK. Feasibility of Conducting a Virtual Exit Exam in Neurosurgery During the SARS-COV19 Pandemic. Neurol India 2021;69:698-702

How to cite this URL:
Salunke P, Sahoo SK, Chacko AG, Baishya BK, Tripathi M, Chabbra R, Karthigeyan M, Aggarwal A, Singh A, Gupta SK. Feasibility of Conducting a Virtual Exit Exam in Neurosurgery During the SARS-COV19 Pandemic. Neurol India [serial online] 2021 [cited 2021 Jul 24];69:698-702. Available from: https://www.neurologyindia.com/text.asp?2021/69/3/698/319207




The neurosurgery residency program is designed to impart clinical and surgical skills to trainees enabling them to manage patients safely. Traditionally, the residents are periodically evaluated for both clinical and surgical skills.[1],[2] This “internal” evaluation plays a vital role in ensuring that residents have the confidence to practice independently. At the end of the training, an exit examination allows for external experts, along with internal examiners to evaluate the trainees, thus avoiding possible bias and prejudice as well as ensuring a fairly standard level of neurosurgical competence across the country. However, the current COVID pandemic has put limitations on such an evaluation methodology where direct patient interview, clinical examination, and surgical skills assessment in the operation theatre are not possible. Here we discuss the preparations for and the conduction of an online clinical exit examination conducted at our institute during the lockdown period of the SARS-COV19 pandemic for six candidates who completed their 3-year neurosurgery training after 3 years of general surgery. Based on this experience, we propose an objective skills assessment module, using the online platform.


 » Materials and Methods Top


Six residents completed their neurosurgery training and appeared for the exit exam in June 2020. The first part of assessment included written examinations covering neuroanatomy, pathophysiology; operative management and recent advances The second part comprised of clinical and surgical assessments also known as the “practical examination.” Due to COVID pandemic, actual physical examination of patients by candidates was not carried out. However, all components of the assessment as stipulated by the Indian Medical Council were included.

Infrastructure at the examination center

Using an online meeting platform, the seminar room was prepared for clinical evaluation of the trainees. However, only the candidates and the internal examiners were allowed inside the examination room.

A laptop with an internet connection was used for the tele-videoconferencing using “ZOOM” (Zoom Video Communications, Inc. USA). The trainee sat in front of the laptop after the allotted case module was opened and a mouse was provided to navigate through the case module as needed. The projection onto the larger screen further provided the finer details of clinical images and radiology. The candidate used the laptop's in-built microphone. They were instructed regarding this new pattern, a day before the evaluation. We had additional laptops with uploaded case modules, electricity generators, and an alternate internet facility as back up.

External examiners

Two neurosurgeons from two separate institutes elsewhere in the country along with two from our own institute conducted the assessment. The external examiners were provided with details of how to log in and to access the online platform and the pattern of the examination one week prior. They were requested to keep themselves free from 9 am to 5 pm for these two days and to use a device with audio-visual facilities for this purpose. Additionally, the examiners were asked to prepare interesting radiological images that could be shared at the time of assessment. Three candidates were evaluated on each day.

Case descriptions

The case descriptions were prepared by faculty members based on the actual clinical history and examination findings of patients. The relevant examination findings, recorded in the form of clinical images (jpeg) and videos (mp4), were digitized and stored with the clinical details in a power point presentation as case modules. The long clinical cases included detailed clinical descriptions with videos demonstrating clinical signs. The short clinical cases were prepared with brief findings and relevant videos/images [Figure 1] and [Figure 2]. Radiology was added to the operative case modules. One long clinical case, two short clinical cases and one operative case were created for each candidate. Each folder had at least one cranial and one spinal case scenario. The aim was to evaluate the residents comprehensively for their clinical knowledge, decision making, and technical efficiency.
Figure 1: Screen shot of slide sorter – case module for long case: Slides were sequentially presented (yellow arrows) to candidates. (a) Clinical symptoms of the patient. (b) Patient images both front and back and video showing patient's gait. (c) Videos demonstrating deep tendon reflexes. (d) Radiology (depicting the pathology)

Click here to view
Figure 2: Screen shot of slide sorter – case module for short case: (a) Clinical images of the patient. (b) Hormonal profile of the patient. (c and d) computed tomogram and magnetic resonance imaging of the brain showing corresponding pathology. The candidates were shown slide in sequential manner (yellow arrows) and were asked corresponding questions

Click here to view


The external examiners were blinded to the clinical presentations and radiology so as to minimize the bias.

Residents were evaluated on the following parameters:

  • Resident's ability to interpret and analyze the clinical data
  • Ability to interpret the clinical image or the video for relevant findings
  • Ability to localize the lesion and list the possible clinical differential diagnoses
  • Identification of anatomical structures and extent of pathology on radiology
  • Arriving at a diagnosis and management plan
  • Knowledge of surgical procedure and peri-operative management related to the final diagnosis


Apart from this, images of pathological specimens and radiology were digitized and prepared for discussion.

The examiners were instructed that 40 min were allotted for examining the candidates for the long case, 20 min each for short cases, 30 mins for operative, and 30 mins for radiology sections of the examination. Three candidates were evaluated per day in four sessions. Following the completion of assessment, feedback was obtained from all the examinees as well as the examiners. The feedback from examinee was based on a questionnaire [Table 1].
Table 1: Questionnaire and feedback form

Click here to view



 » Results Top


The audio-visual system arrangement in the examination hall was satisfactory and there were no technical interruptions during these two days. However, on one occasion the candidates had to be prompted to speak loudly into the microphone close to the laptop. As the larger screen was placed up on the wall, the candidates had to look up from the laptop and had some difficulty in manipulating the cursor with the mouse [Figure 3].
Figure 3: Image showing a candidate interacting with the external examiners through the online platform with the use of projected large screen

Click here to view


With regard to the external examiners (BKB) used a cell phone for the entire clinical interview that had very good audio and video quality but he had to switch to the desktop to show his radiology slides resulting in a few minutes disruption. The other examiner (AGC) had poor audio that was rectified to some extent after use of a headphone.

The candidates presented the history that was shared with the examiners so a continuous interaction was possible. The images and clinical videos helped to maintain the flow of interaction and broaden the scope of the discussion. The radiological images enabled detailed assessment of normal and abnormal anatomy., When the individual scores of trainees were collected from all the experts, there was no remarkable difference in the grades awarded by each examiner.

Feedback from the examinees was analyzed after the assessment. All residents were able to interpret the clinical signs from the patients' image or examination video shown in the case modules. Four residents were of the opinion that the clinical data provided was adequate to establish a localization of the lesion. However, two candidates were unable to correlate the clinical data accurately and on further questioning they felt that a direct interview and physical examination of the patient would have helped in establishing the clinical diagnosis better. Four residents rated this method of evaluation nearly equivalent to direct one on one interaction with the examiners. However, two were disappointed due to the lag effect inherent to the online audiovisual system. Overall, the online platform was rated 4 out of 5 by the candidates and all recommended this method of evaluation can be considered in future especially in an unprecedented situation like the current one.


 » Discussion Top


Most neurosurgical training programs impart a core set of clinical knowledge and surgical skills in a time bound manner.[2] Trainees are periodically appraised for the clinical knowledge and operative skills by faculty members of the training institute to enable progression to the next level in the program.[3],[4],[5] The Medical Council of India mandates an exit examination at the end of the postgraduate training program where each trainee is comprehensively evaluated with theory and practical tests in the presence of both internal and external experts from other institutes.

At the time of exit examination, 2-4 cases of the hospital admissions, tested positive for COVID each day. Due to COVID restrictions postponing the exit exam was discussed considering the safety issues. However, the time line for resolution of the COVID pandemic and return to normal was uncertain. It was therefore decided to conduct an online examination. Given the uncertainty this might be the way examinations are to be conducted in future if the pandemic continues.

Setting up of an Online platform system

There are several platforms like Google meeting, Skype, Webex, Microsoft teams, and Zoom commercially available for use. Good audio-visual systems with internet facility are essential for the use of an online platform system. Certain platforms have the advantage of simultaneous access to the same screen, both for the examinee and the examiner so as to make the discussion more interactive. For smooth running of the online platform both the examinee and the examiners must be well informed in advance about the use of audio-visual system. Through this online platform examiners from distant part of the country able to interact with the examinee and continuous discussion can be carried out similar to direct objective evaluation.

A major issue with the online platform is “lag” in audiovisual system which can affect the flow of discussion. Prior knowledge of such “lag” minimizes the communication gap during the assessment.

The candidates should be appropriately dressed and should maintain the usual decorum. They should let the examiners finish with comments or question before speaking into the microphone.

History and examination

Use of original patient data (examination findings and clinical images) makes the case module more authentic and realistic for discussion and avoided classical text book case descriptions. The interpretation of clinical history require a particular level of understanding of the subject. All clinical, laboratory, and radiological data in a case module were presented in a sequential manner similar to what the residents would follow in their routine practice. The external experts too were blinded to the case scenarios so as to make the assessment more interesting and unbiased. Videos depicting a particular sign like pendular knee jerk, ankle clonus, spastic gait, Romberg sign, etc., were incorporated to add objectivity to the evaluation process. The videos of nystagmus, internuclear ophthalmoplegia, lobar signs, fasciculations, cerebellar signs, etc., can be incorporated in addition to pictures of fundus, muscle wasting to name a few. The chronological appearance of all these clinical details further helps the trainee to draw a mental picture of the patient and can correlate all the findings to reach up to a diagnosis. Four out of six residents successfully picked up the clinical clues depicted in the case modules and were able to make definite diagnosis.

Advantages of traditional methods – History taking is an art and this requires interacting with the patient. Similarly, listing the chronology of symptoms requires some sort of deliberation. Observing the candidate while directly examining and eliciting clinical signs has the advantage of evaluating his applied clinical knowledge which can be cross verified. Simultaneously the trainee can read the body language of the examiners and can judge whether their analysis is correct and can avoid unnecessary mistakes.

Testing all the patients, examinees, and internal examiners before the assessment may provide an opportunity for direct clinical evaluation which can be telecasted to the examiners by an overhead camera system. Needless to say that personal protective equipment is essential for conducting such evaluation. As suggested by one of the external experts, an “actor” can be included to assess the clinical acumen. The actor here could be one of the fellow residents or a faculty member.

Radiology/pathology specimen

Through the online platform the candidate and the examiner can view the radiology and imaging of specimen. This will allow a continuous interaction like that of the direct interview. The online platform with the sharing of the screens also enables testing the candidate's ability to identify verbally specific anatomic landmarks pointed out by the examiner or himself pointing out to landmarks as asked by the examiner.

Operative neurosurgery

Operating skills acquisition is the key step in neurosurgery training. Several tools have been devised and validated to evaluate them objectively.[5],[6],[7],[8] Many centers of India and almost all in abroad have done away with surgical skills assessment in the presence of external experts due to high anxiety level of the candidates, unavailability of suitable cases, and ethical justifications.

Most centers have adopted operative skill assessment in the operating room where the trainee performs surgical procedures directly on patients. A real time scoring and feedback is considered as the best way of assessing residents for technical skills like instrument handling, tissue respect, dexterity, flow of surgery, and knowledge of a specific procedure. Here we have formulated case scenarios for tumor localization and provided the radiology with a skull bone for discussion of relevant surgical planning and peri-operative issues. For better objectivity, operative video scan be used where residents can be evaluated for knowledge of surgical anatomy, operative steps, intra-operative complications, and their management. Alternatively, video recordings of the surgeries performed by the trainees can be used for discussion. Then examiners have the opportunity to assess the operative skills of the trainee by observing that particular surgery step by step similar to the “procedure based assessment” in the operating theater.[4],[9] Neurosurgical centers with skills laboratory can also use operative modules (cadaveric/non-cadaveric) for such technical skills evaluation. Assessment directly in the skills laboratory provides a similar atmosphere like that of the operating theatre where the trainee can be evaluated for knowledge of instruments and operative procedures. This appears to be more practical and also feasible with the use of online platforms.

Limitations

Online audiovisual system has some lag effect and may disturb the flow of discussion. Clinical case modules directly demonstrate some signs which cannot be cross verified either by the examinee or by the examiner. This methodology deprives the candidates from the advantage of reading the examiner's body language that gives important clue for avoiding mistakes.

Finally, one tends to draw an analogy from the fact that evaluation of an aviation pilot cannot be made on a simulator alone. It is difficult to assess a surgeon's skills based on these modules alone.


 » Conclusion Top


Conducting virtual assessment for neurosurgery residents using online platforms with an aim of objective evaluation of clinical and surgical skills is challenging. Case modules incorporated with actual patient data, clinical imaging, videos demonstrating clinical signs, actors, operative videos, and evaluation in the skills laboratory makes it more practical and realistic in such unprecedented times.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
 » References Top

1.
Hadley C, Lam SK, Briceño V, Luerssen TG, Jea A. Use of a formal assessment instrument for evaluation of resident operative skills in pediatric neurosurgery. J Neurosurg Pediatr 2015;16:497-504.  Back to cited text no. 1
    
2.
Jensen RL, Alzhrani G, Kestle JRW, Brockmeyer DL, Lamb SM, Couldwell WT. Neurosurgeon as educator: A review of principles of adult education and assessment applied to neurosurgery. J Neurosurg 2017;127:949-57.  Back to cited text no. 2
    
3.
Doyle JD, Webber EM, Sidhu RS. A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 2007;193:551-5.  Back to cited text no. 3
    
4.
Beard JD. Assessment of surgical skills of trainees in the UK. Ann R Coll Surg Engl 2008;90:282-5.  Back to cited text no. 4
    
5.
Sarkiss CA, Philemond S, Lee J, Sobotka S, Holloway TD, Moore MM, et al. Neurosurgical skills assessment: Measuring technical proficiency in neurosurgery residents through intraoperative video evaluations. World Neurosurg 2016;89:1-8.  Back to cited text no. 5
    
6.
Larson JL, Williams RG, Ketchum J, Boehler ML, Dunnington GL. Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents. Surgery 2005;138:640-9.  Back to cited text no. 6
    
7.
Meterissian SH. A novel method of assessing clinical reasoning in surgical residents. Surg Innov 2006;13:115-9.  Back to cited text no. 7
    
8.
Mitchell EL, Arora S, Moneta GL, Kret MR, Dargon PT, Landry GJ, et al. A systematic review of assessment of skill acquisition and operative competency in vascular surgical training. J Vasc Surg 2014;59:1440-55.  Back to cited text no. 8
    
9.
Wohaibi EM, Earle DB, Ansanitis FE, Wait RB, Fernandez G, Seymour NE. A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance. J Surg Educ 2007;64:333-41.  Back to cited text no. 9
    


    Figures

  [Figure 1], [Figure 2], [Figure 3]
 
 
    Tables

  [Table 1]



 

Top
Print this article  Email this article
   
Online since 20th March '04
Published by Wolters Kluwer - Medknow