Introduction

For fourth-year medical students, the COVID-19 pandemic limited learning experiences, altered professional development opportunities, introduced virtual interviews, and cancelled the “away rotation” or visiting sub-internship.1,2 Medical educators had already been scrutinizing the final year of medical school due to learning inefficiencies and concerns for residency unpreparedness.3 In fact, some had encouraged the reformation of visiting sub-internships to provide more educational benefit based on the investment they require.4 Given safety concerns during the COVID-19 pandemic, the Coalition for Physician Accountability5 and other national organizations6 issued recommendations to cancel away rotations for the 2020–2021 residency application cycle. Thus, the COVID-19 pandemic offered an opportunity to reimagine how clinical departments could remotely meet the goals of the traditional visiting sub-internships: advanced medical student clinical training, mentorship, career exposure, and an introduction to residency life. We describe herein how we created a virtual space to meet some objectives of the traditional visiting sub-internship for Otolaryngology-Head and Neck Surgery (OHNS).

Methods

We first informally interviewed University of Michigan third-year medical students applying into OHNS, residents, and program leadership about the objectives of traditional visiting sub-internships. Based on these results, we designed a free, one-day virtual event to meet three objectives: (1) exposure to the breadth and depth of OHNS sub-specialties, (2) introduction to the University of Michigan OHNS department and residency program, and (3) compilation of resident-recommended didactic content for each OHNS sub-specialty. The event was held on June 13, 2020, and featured sessions focused on OHNS sub-specialties as well as specific topics such as scholarship during residency and diversity, equity, and inclusion (Table 1).

Table 1.

Event Agenda

Main Topic: Additional topics discussed

Part I

Welcome to Michigan Medicine

10:00 AM–10:20 AM

Day in the Resident Life: Program Director Address (Philosophy, Teaching, Teamwork, Program Strengths), Introduction of Residents (Background, Location Choice), Description of Each Year, OR/Clinical Experience, Community

10:20 AM–11:15 AM

Head and Neck Oncology: Faculty Representatives-Specialty Overview, Clinical Experience, Leadership, Research

11:15 AM–12:00 PM

Neurotology: Faculty Representatives-Specialty Overview, Clinical Experience, Research, Temporal Bone Lab

12:00 PM–12:20 PM

Michigan Medicine Alumni: Alumni Map, Personal Experience of Two Alumni Representatives

12:20 PM–12:40 PM

Break

12:40 PM–1:00 PM

Part II

Pediatric Otolaryngology: Faculty Representatives-Specialty Overview, Medical Education, Simulation

1:00 PM–1:30 PM

Scholarship at Michigan: Faculty Representatives-Research Support, Grant Funding, Quality Improvement, Innovation, Simulation, Resident Research Experience

1:30 PM–2:00 PM

Diversity Equity and Inclusion: Health Equity Introduction (Racism, Social Determinants of Health, Disparities), Cultural Humility, Advocacy

2:00 PM–2:30 PM

Laryngology, Skull Base, and Facial Plastics: Faculty Representatives-Specialty Overview, Rotation/Team Structure, Off-Site Clinical Experiences

2:30 PM–3:00 PM

How to Ace the Remote Application Cycle: Challenges/Opportunities, Interview Practice, Resources (AAMC, Headmirror), Tips/Tricks, Personal Statement, Re-applicants, Additional Questions

3:00 PM–3:30 PM

Invited faculty were asked to share aspects of their daily practice and insights on how they integrate leadership, educational philosophy, and scholarly activity into their careers. Similarly, resident presenters were chosen to represent junior- and senior-level trainees who were involved in a wide range of pursuits both inside and outside of the residency program. Each session was structured so that a resident moderator would introduce and then interview the presenters. Question and answer sessions were held in real time. All participants were sent guidelines for the event, presenter contact information, and electronic resources and book recommendations curated by University of Michigan OHNS residents.

Advertisement for the program took place through OHNS medical student forums, institutional diversity champions, social media, and word of mouth. The event was hosted as a Zoom webinar with two residents as moderators. Following the program, surveys were distributed to all participants and panelists through Qualtrics. Using 5-point Likert scale questions, participants were asked about the helpfulness of the program for learning about the scope of the field, University of Michigan residency, and OHNS electronic and book resources, as well as the program length and likelihood of recommending the program to other students.

Participant demographics and duration of attendance data were collected. Descriptive statistics were calculated for Likert scale questions on Qualtrics software. Two of the authors (JLF and EJM) created, combined, and reconciled themes for free response questions. The University of Michigan Institutional Review Board deemed this study exempt (HUM00182084).

Results

There were 448 registrants and 327 final participants, representing 138 institutions, 34 states and territories, and 10 countries. The majority (90%) of participants were senior medical students (fourth year only), with fewer junior medical students (6%), residents (3%), and participants with unknown training status (1%). The event had a total of 5 content hours, and participants on average were signed into the event for 4.1 hours (standard deviation 2.2).

There were 102 respondents to the participant survey (31% response rate). Of respondents, 99% stated that they would be very likely (79%) or somewhat likely (20%) to recommend the program to another student. Approximately half of participants felt the event was an appropriate length (n = 48, 47%) or a little too long (n = 46, 46%). The majority of participants found the event helpful for learning about the scope of the field (96%), the residency program (97%), and specialty-specific resources (86%) (Table 2). Themes extracted from free responses showed that participants found hearing from residents (n = 44, 43%) and faculty (n = 42, 41%) as the most helpful aspects of the program. A smaller number of responses noted description of program opportunities (n = 25, 25%), tips for the application process (n = 19, 19%), and live question and answer (n = 11, 11%) as the most helpful parts of the event. Suggestions for event improvement included more resident interaction (n = 15, 19%), a shorter event or more spread out programming (n = 13, 17%), and integrating more information regarding the application process (n = 12, 16%).

Table 2.

Participant Likert Scale Question Responsesa

Participant question

Mean Likert score (SD)

Number responses of very or somewhat helpful/likely (%)

How helpful was the program for learning about the scope of Otolaryngology-Head and Neck Surgery as a field?

4.6 (0.6)

98 (96)

How helpful was the program for learning about the University of Michigan/Michigan Medicine residency program?

4.8 (0.5)

99 (97)

How helpful was the program for learning about Otolaryngology-Head and Neck Surgery electronic and book resources?

4.3 (0.8)

88 (86)

How likely would you be to recommend this program to another student?

4.8 (0.4)

101 (99)

    Abbreviation: SD, standard deviation.

  • aResponses were measured on a 5-point scale, with 0 = not at all helpful to 5 = very helpful, or 0 = not at all likely to 5 = very likely.

Participants stated that their biggest concern for the upcoming application cycle was getting to know programs in a remote application cycle (n = 25, 28%), interviews in a virtual setting (n = 23, 26%), the lack of away rotations (n = 18, 20%), and their individual competitiveness (n = 14, 16%), including students who felt their educational and career development opportunities had been limited by the pandemic (n = 7, 8%). Students also noted potential disadvantages for special applicant populations (n = 13, 15%), including international medical students, osteopathic students, students participating in the couples match, and students from institutions without OHNS departments.

As further described through free response, participants thought the most difficult aspect of a residency program to evaluate remotely was the program’s or institution’s culture (n = 65, 76%), followed by the program’s city (n = 22, 26%) or facilities (n = 16, 19%). Students were also concerned about the ability to assess their personal fit in a program (n = 16, 19%) and difficulty getting to know residents remotely (n = 11, 13%).

Discussion

Visiting medical student sub-internships, despite their cost and time investment, are a common and potentially vital portion of the transition to residency in OHNS and other specialties.4 With the notable limitation of visiting sub-internships during the COVID-19 pandemic, the University of Michigan OHNS department created a one-day virtual event to address some of the barriers to educational experiences.

We found that this event was well-received by participants and engaged a broad international audience of 327 participants in a field that offers only 350 new residency spots through the National Resident Matching Program annually.7 While visiting sub-internships have resumed, events such as the one described herein may still have a place as an introduction to the field or as a complement to mentorship programs for premedical students, early medical students, international students, and those without home residency programs. Earlier exposure to various residency programs may also lead to more informed application and interview decision-making, which would be particularly valuable if virtual interviews continue.

Even though participants on average stayed signed-in for the majority of this event, the program was not designed to measure direct engagement. While the majority of participants stated that the event length was just right or just a little long, we suggest that future events be shorter or more spread out, as greater adoption of similar experiences may contribute to virtual burnout.8,9 Additionally, at the time of the event, students were generally not involved in clinical activity given guidance from the Association of American Medical Colleges,10 but at the time of writing this report, clinical responsibilities for students had resumed. The length and timing of future events should be planned accordingly. Evaluation of this program was limited by a 31% survey response rate and resulting selection bias of respondents. Additionally, the survey was designed to assess participant perceptions of whether program objectives were met and not the actual knowledge gained.

This program did not allow for in-depth didactic learning or breakout room discussion, compared to other multiple-week virtual programs.1115 Nonetheless, the majority (82%) of participants identified resident interaction as the most valuable aspect of the program. Given the specific concern of evaluating a program’s culture remotely per 76% of survey respondents, future events should create interactive experiences between faculty, residents, and participants. At our institution, we have started an initiative that pairs University of Michigan residents and faculty with senior medical students from across the country to discuss transition to residency topics and high-yield case-based reviews. While this program facilitates social networking and small group interaction, it does not replace individualized mentorship, meaningful interaction leading to a letter of recommendation, or other aspects of in-person visiting sub-internships. We encourage the development and continued support of specialty-wide virtual offerings to enhance didactic learning,15 create a national network of faculty and resident mentors,16 and provide other asynchronous virtual career development opportunities for students to further explore the field. For instance, Headmirror (www.headmirror.com), a website designed to provide “a centralized high-quality open-access [OHNS] resource,” has recently established a National Otolaryngology Interest Group that may simultaneously engage a wider student community and supplement sub-internships.

The COVID-19 pandemic has afforded the opportunity to create innovative educational experiences, such as the one described herein. Similar offerings could enhance or replace the traditional components of the final year of medical school training, such as the visiting sub-internship.17 As we move beyond the COVID-19 pandemic, medical educators should continue to critically examine and reimagine the sub-internship experience as well as the entire transition to residency process.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author Roles

Janice L. Farlow: design, participation, data collection, analysis, presentation; Emily J. Marchiano: design, participation, data collection, analysis, presentation; Andrew J. Neevel: presentation, manuscript review; Jeffrey S. Moyer: participation, manuscript review; Marc C. Thorne: participation, manuscript review; Lauren A. Bohm: design, participation, presentation, manuscript review.

References

1. Rose S. Medical student education in the time of COVID-19. JAMA. 2020; 323(21):2131–2132. doi:10.1001/jama.2020.522710.1001/jama.2020.5227

2. Medical student away rotations for the remainder of 2020–21 and 2021–22 academic year. Association of American Medical Colleges. Published December 13, 2021. https://www.aamc.org/what-we-do/mission-areas/medical-education/away-rotations-interviews-2020-21-residency-cyclehttps://www.aamc.org/what-we-do/mission-areas/medical-education/away-rotations-interviews-2020-21-residency-cycle

3. O’Brien BC. What to do about the transition to residency? Exploring problems and solutions from three perspectives. Acad Med. 2018; 93(5):681–684. doi:10.1097/ACM.000000000000215010.1097/ACM.0000000000002150

4. Griffith M, DeMasi SC, McGrath AJ, Love JN, Moll J, Santen SA. Time to reevaluate the away rotation: improving return on investment for students and schools. Acad Med. 2019; 94(4): 496–500. doi:10.1097/ACM.000000000000250510.1097/ACM.0000000000002505

5. The Coalition for Physician Accountability. Final Report and Recommendations for Medical Education Institutions of LCME-Accredited, U.S. Osteopathic, and Non-U.S. Medical School Applicants. May 11, 2020. https://www.aamc.org/system/files/2020-05/covid19_Final_Recommendations_05112020.pdfhttps://www.aamc.org/system/files/2020-05/covid19_Final_Recommendations_05112020.pdf

6. SUO/AADO/OPDO statement regarding away rotations. Society of University Otolaryngologists, Association of Academic Departments of Otolaryngology, Otolaryngology Program Directors Organization. Published April 27, 2020. https://cdn.ymaws.com/suo-aado.org/resource/resmgr/covid-19/suo-aado-opdo_away_rotation_.pdfhttps://cdn.ymaws.com/suo-aado.org/resource/resmgr/covid-19/suo-aado-opdo_away_rotation_.pdf

7. National Resident Matching Program, Washington DC. National Resident Matching Program, Results and Data: 2020 Main Residency Match. May 2020. https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2020/06/MM_Results_and-Data_2020-1.pdfhttps://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2020/06/MM_Results_and-Data_2020-1.pdf

8. Samara O, Monzon A. Zoom burnout amidst a pandemic: perspective from a medical student and learner. Ther Adv Infect Dis. 2021;8. doi:10.1177/2049936121102671710.1177/20499361211026717

9. Yu JL, Cho DY, DeSanti RL, Kneib CJ, Friedrich JB, Colohan SM. Resident experiences with virtual education during the COVID-19 crisis. J Plast Reconstr Aesthet Surg. 2021; 74(8):1931–1971. doi:10.1016/j.bjps.2021.05.03710.1016/j.bjps.2021.05.037

10. Important guidance for medical students on clinical rotations during the coronavirus (COVID-19) outbreak. Association of American Medical Colleges. Published May 17, 2020. https://www.aamc.org/news-insights/press-releases/important-guidance-medical-students-clinical-rotations-during-coronavirus-covid-19-outbreakhttps://www.aamc.org/news-insights/press-releases/important-guidance-medical-students-clinical-rotations-during-coronavirus-covid-19-outbreak

11. Landeen KC, Esianor B, Stevens MN, et al. Online otolaryngology: a comprehensive model for medical student engagement in the virtual era and beyond [published online ahead of print July 5, 2021]. Ear Nose Throat J. doi:10.1177/0145561321102974810.1177/01455613211029748

12. Bernstein JD, Ball LL, Nardone ZB, Watson D. A virtual sub-internship for otolaryngology-head and neck surgery. Laryngoscope Investig Otolaryngol. 2021; 6(5):952–957. doi:10.1002/lio2.63710.1002/lio2.637

13. Chao TN, Frost AS, Brody RM, et al. Creation of an interactive virtual surgical rotation for undergraduate medical education during the COVID-19 pandemic. J Surg Educ. 2021; 78(1):346–350. doi:10.1016/j.jsurg.2020.06.03910.1016/j.jsurg.2020.06.039

14. Lee J, Stevens MN, Landeen KC, Lipscomb BE, Whigham AS. A four-week otolaryngology virtual educational curriculum for prospective applicants [published online ahead of print October 24, 2021]. Ear Nose Throat J. doi:10.1177/0145561321105233710.1177/01455613211052337

15. Steehler AJ, Pettitt-Schieber B, Studer MB, Mahendran G, Pettitt BJ, Henriquez OA. Implementation and evaluation of a virtual elective in otolaryngology in the time of COVID-19. Otolaryngol Head Neck Surg. 2021; 164(3):556–561. doi:10.1177/019459982095115010.1177/0194599820951150

16. Farlow JL, Devare J, Ellsperman SE, et al. Virtual resident mentorship groups for fourth year medical students applying into otolaryngology-head and neck surgery [published online ahead of print May 12, 2021]. Ann Otol Rhinol Laryngol. doi:10.1177/0003489421101574010.1177/00034894211015740

17. Farlow JL, Marchiano EJ, Fischer IP, Moyer JS, Thorne MC, Bohm LA. Addressing the impact of COVID-19 on the residency application process through a virtual subinternship. Otolaryngol Head Neck Surg. 2020; 163(5):926–928. doi:10.1177/19459982093477510.1177/194599820934775