Factors Infl uencing Underrepresented Minority Applicants’ Ranking of General Surgery Residency Programs

Background: Underrepresented minorities (URM) are disproportionately lacking in academic surgery, yet there is wide variation in URM in residency programs. Th is mixed methods study aims to describe factors that URM interns identifi ed as important in their rank list of general surgery residency programs. Th ese data will aid institutions in increasing URM representation and creating supportive training environments in academic surgery. Methods: We employed a mixed methods approach using an online survey and semi-structured interviews developed through focus group engagement and literature review. Interviews were audio recorded, transcribed, and coded for themes. We conducted narrative thematic analysis.


Introduction
5][6] Despite the increasing diversity of both the national population and the population of matriculating medical students, only 7% of academic surgeons in the United States identify as URM, and there has been no change in this number from 2005 to 2018. 7ncreasingly, surgery departments across the United States are implementing strategies to diversify their residency classes. 8However, residency programs show differing rates of success.Studies regarding residency program rank lists, in general, show that overall location and program factors (eg, morale and clinical experience) are the most influential factors in rank list for all applicants. 9,10Women and URMs are more likely also to prioritize family and diversity factors. 9,10However, factors specific to URM applicants pursuing general surgery programs remain unknown and understudied. 11o increase URM recruitment and retention in surgical disciplines, programs must have a deeper understanding of factors important to URM candidates applying into general surgery residency.In this study, we explored factors considered by URM in creating their National Resident Matching Program (NRMP, or Match) rank order list for general surgery residency.We hope these results yield opportunities for surgical residency programs to strengthen their cohorts by improving diversity and representation in pursuit of racial justice and health equity.Specifically, we hypothesize that URM participants will prioritize factors related to diversity and inclusion efforts.

Participants
We employed a mixed method approach using an online survey and semi-structured interviews developed through focus group engagement and literature review.Participants were eligible for the survey if they matched during the 2020 cycle into a US general surgery residency program regardless of URM status.URM status was self-reported.Surveys and interviews were conducted from June 2020 to July 2020 prior to matriculation at any US general surgery residency program.Interview participants were exclusively URM, defined as Black/African American, Hispanic/Latino, Native Hawaiian/Pacific Islander, and/or Native American.Participants were ineligible if affiliated with a Historically Black College/University because these programs have traditionally accepted more URM residents than other programs.This study was exempted by the University of Michigan Institutional Review Board (HUM00183224).

Survey
An electronic 18-question survey was distributed to URM and non-URM general surgery interns through targeted social media platforms and emails to determine important factors for ranking residencies.Both the survey questions and the interview guide were developed by identifying important factors and themes for ranking residency programs via (1) a focus group of current URM medical students and faculty at the University of Michigan and (2) an extensive literature review from PubMed searches from 2004 to 2020 combining the keywords "URM," "general surgery," and "residency."Surveys evaluated 18 pre-determined factors contributing to rank lists focused on (1) program diversity features, (2) educational factors, (3) prior and anticipated experiences, and (4) lifestyle factors.Participants were provided two optional write-in factors if they wanted to report a factor not part of the 18 pre-determined factors (Table 1).Differences in survey responses were evaluated using Mann-Whitney U tests with an alpha of 0.05.Bold questions were asked of all respondents.Other questions were suggestions for guiding conversation according to relevant themes.

Survey Results
Sixty-four students (median age 25-29, 67% female; Table 3) out of 993 total students who matched to general surgery residency programs in the NRMP in 2020 completed the survey. 12here were 22 URM respondents, 12 of whom self-identified as Black or African American, 7 as Hispanic or Latino, 2 as Pacific Islander, and 1 as Native American or Alaska Native (Table 3).
Of the 18 factors contributing to general surgery residency rank, post-graduate job or fellowship opportunities, the feeling of being wanted at the program, anticipated clinical experience, and perceived morale of residents were ranked highest among all participants.Of the list, URM residents ranked post-graduate opportunities and perceived morale of residents the highest of all factors.URM participants ranked the importance of factors related to diversity (diversity/ inclusion statements, resident/faculty diversity), education (post-graduate/fellowship opportu-

Table 2. (Continued)
 open access -michjmed.orgnities, academic reputation, program size, revisit opportunities), experiences (mentorship, feeling of being wanted, support of resident-led initiatives), and lifestyle (benefits/finances, spouse job opportunities, political climate) higher than did non-URM participants (p<0.05; Figure 1).Additionally, URM participants ranked certain program educational factors-post-graduate opportunities (p<0.05),academic reputation (p<0.05),program size (p<0.05),and revisit opportunities after interview day (p<0.05)-higherthan did non-URM participants (Figure 2).URM participants prioritized a supportive learning environment during residency more than did their non-URM peers.They prioritized factors such as formal mentorship opportunities (p<0.001), the feeling of being wanted (p<0.05), and department support of resident-led initiatives (p<0.05; Figure 3).Similarly, URM participants placed greater emphasis on residency experience outside of academics, such as program benefits and finances (p<0.05),job opportunities for one's spouse or significant other (p=0.05),and the political climate of the program (p<0.05; Figure 4).Some factors did not differ in importance between URM and non-URM participants.Among these factors were proximity to family, geographic location, anticipated clinical experience, and perceived morale of residents.Overall, URM participants rated all factors higher than did non-URM participants.
 open access -michjmed.org open access -michjmed.orgactively supported resident-led initiatives by providing resources, encouragement, and assistance in project development.All interviewees weighed the experiences of current URM residents heavily in making their rank lists.Stories and opinions by current URM residents were a significant factor in an interviewee's perception, both positive and negative, of a program.
Active inclusivity efforts were highlighted across all interviews as critical to evaluating whether a program would be a good place to train as a URM.Though efforts and initiatives were different across programs, incoming interns consistently looked for concrete plans and actions to increase representation and inclusion.Interviewees distinguished whether interest in supporting diversity was genuine by looking for program recognition of diversity issues as important, willingness to discuss issues candidly, and articulation of concrete plans to ameliorate disparities."They actually had a good number of Black residents across their entire residency classes.[. ..]I met a couple at the pre-interview dinner, and then one or two during the interview day, and after my interview, that program director literally sent an email contact and also CC' ed me on introductory emails with other residents of color, which I thought was really, really nice."(ID 2) "I asked, you know, kind of the tougher questions -just asked [current residents], ' As a female, have you ever felt like you were treated differently in your training class?'I asked if they had any mentors either that were female or people of color that they felt comfortable going to." (ID 5)

Discussion
This study identified several discrete factors that URM interns value in the NRMP rank list process, including (1) diversity factors, (2) active inclusivity efforts, and (3) a supportive learning environment.The demographic distribution of age, race, and other variables in the cohort described in this study is consistent with the distribution for incoming general surgery interns in 2020 as reported by the Association of American Medical Colleges (AAMC). 12Our data align with that of other studies, highlighting that all incoming interns prioritize post-graduate opportunities, anticipated clinical experience, perceived morale of residents, and the feeling of being wanted. 9,10However, our data demonstrate unique factors valued by URM during the rank list process.

Table 4. (Continued)
 open access -michjmed.orgFirst, diversity factors were especially important to URM medical students applying into general surgery.URM interns cited the "diversity tax" as a concern related to the lack of persons of color in a residency program.The diversity tax describes the phenomenon that URM academics are frequently asked to take on additional, uncompensated work to address systemic racism at their institutions, such as serving on diversity committees and mentoring URMs in their field. 13When there are very few URMs in a program, this burden rests on the shoulders of just a few individuals.Understandably, incoming interns did not want to be put in the position of speaking on behalf of their entire race.Facing an academically and clinically rigorous program such as general surgery residency, URM interns unsurprisingly would be hesitant to take this on.
URM general surgery interns reported using visual scans of representation on program websites and on interview day as a proxy for lived values of diversity and inclusion because assessment was otherwise difficult.General surgery programs could be more welcoming to URM applicants through appropriate representation in the faculty ranks and by increasing visibility of diverse faculty on interview day and online.Rather than shying away from issues of diversity, we suggest programs lean into difficult conversations by being intentional, frank, and public with goals to increase representation and diversity of perspective.This approach aligns with URM applicants' desire to see programs willingly discuss related efforts candidly.
Second, incoming interns also looked for evidence of inclusion initiatives.Several incoming interns expressed that some programs seemed to feel, as one participant put it, "behind on the diversity game" and thus expressed interest in URMs to increase their diversity numbers as opposed to genuinely valuing diverse perspectives.Being told that a program values diversity is not as reassuring to URMs as seeing evidence of action to promote inclusivity because of the risk that a program is a performative ally. 14Programs can signal true allyship in a way that is valuable to URM applicants by funding meaningful initiatives to address self-identified areas for improvement.
Third, the data emphasize that a supportive program environment is highly valued by URM interns.The availability of formal mentorship opportunities was a significant factor in evaluating programs for URM interns.URM interns also looked for programs that would actively support their goals.This support may come in the form of resources such as allocated funding and assistance with project development through goal-specific mentorship or protected time.Several interns noted specific interests within academic surgery and shared examples of their interviewers describing how a particular program could help them achieve goals in their area of interest.
Our data indicate that URM applicants weigh the presence of a visibly diverse and inclusive culture and facilitated mentorship opportunities heavily when making decisions about where to train for general surgery residency.Programs should lean into difficult conversations by being transparent and realistic regarding efforts to increase representation and diversity within their departments.By valuing and allocating resources toward resident-led goals, programs can create an environment that is empowering and inclusive, thus providing a supportive training environment for URM residents.This environment would benefit not only the professional growth and development of URM residents but also the surgery community at-large through diversity of thought and improved patient care.
Our study is not without limitations.One limitation is the number of interviews and surveys completed relative to the number of applicants into general surgery.Distribution was limited to targeting professional social media, mainly what was then known as Twitter.As a result, we were unable to assess response rate.Additionally, given the COVID-19 pandemic, graduating students in the United States may have been in various phases of being recruited earlier to assist with patient care during our study, so there may be a selection bias in participants who had the bandwidth to  open access -michjmed.orgcomplete interviews.Similarly, interviewees during this cycle were the last to complete interviews in-person to date.It is possible applicants have different perceptions of programs' diversity initiatives during in-person versus virtual interviewing.Given that rank lists were submitted prior to the height of the pandemic, it is unlikely the COVID-19 pandemic affected participants' perspectives throughout the interviewing and recruiting process.Lastly, our findings are mostly applicable to underrepresentation by race and ethnicity given the lack of responses encapsulating experiences of people outside the gender binary.Despite these limitations, we believe that our study provides important, previously unknown information given that currently only 8.9% of general surgery residents self-identify as Black, African American, Hispanic, or Latinx. 9More research is needed on the perspective of program directors and their approach to this issue.

Conclusions
This study indicates that incoming URM general surgery interns weighed several discrete factors more heavily than did incoming non-URM interns in building their Match rank order list.These factors are (1) diversity issues including representation within the department, (2) active inclusivity efforts, and (3) a supportive program environment.To foster a more diverse residency, programs should address these factors directly through (1) elevating minority faculty and making representation a department-wide priority, (2) funding meaningful initiatives to address areas for improvement in the diversity space, and (3) listening to and supporting resident-led goals.These findings are critical for making the field of academic surgery more welcoming to URM applicants.Programs must recognize diversity issues as important; discuss them candidly; and make it a priority to address disparities directly, intentionally, and publicly.

Figure 1 .
Figure 1.Diversity Factors in Ranking of Programs All factors shown were significantly different (p<0.05).Median participant rating of factor importance via Likert scale shown.

Figure 3 .
Figure 3. Experience Factors in Ranking of Programs All factors shown were significantly different (p<0.05).Median participant rating of factor importance via Likert scale shown.

Figure 4 .
Figure 4. Lifestyle Factors in Ranking of Programs All factors shown were significantly different (p<0.05).Median participant rating of factor importance via Likert scale shown.

Table 1 .
Survey Questions 1. Did you match into general surgery in March 2020?

Table 2 .
Semi-Structured Interview Guide 1. What is your age? 2. What is your marital status? 3. Do you have any children?4. What gender do you identify as? 5. What ethnicity do you identify as? 6.What race do you identify as? Program-specific characteristics Tell me about your residency application experience.1.What were the most important things you were looking for in a residency program? 2. In making your rank list, how did the factors you mentioned earlier influence your order? 3. Was patient population a factor?4. Were there any factors that you were conflicted with in deciding your ranking? 5. Are those factors still most important to you in retrospect?

Can you tell me about your interview day at the program you matched at? 1
. If the program you matched at was not your number 1 rank, could you tell me about your interview day at the program you ranked first? 2. Did anyone from the program you matched at reach out to you after your interview day?How did you feel about this?Was this important in creating your rank list? 3. On your interview day, did you see or interact with any minority residents?Faculty?Or patients?Tell me more about these interactions if you did. 4. If program diversity was an important factor, how did you assess that?InterviewA snowball sampling technique was used to identify incoming URM students in the 2020 Match class to complete interviews (Table2).URM interns were interviewed through semi-structured telephone interviews.We chose to interview only URM, as we aimed to highlight their voices with this study.Interviews were recorded, transcribed, independently coded by two researchers, and analyzed for themes.
5. Did you talk to other URM applicants about this program and did this influence your decision?Are there any other underrepresented minorities in your intern class?(Continued )  open access -michjmed.orgConstruct/

are your long-term goals? How does the program you matched in factor into those goals
? 1. Can you tell me a bit more about where you grew up? 2. Anyone else in your family attend college?Medical school?Have a graduate degree?3.Where does your family live currently?Can you tell us about your support system in the town your new program is located? 4. What are your research or service interests?

Table 4 .
Identified Themes From Semi-Structured Interviews strongly in solidarity with other URMs.Like you want to warn someone if you are hating it and you want to lure people if you're loving it."(ID 1) "I knew that I did not want to be the only Black person in my program.I wanted to be at a program that truly, truly valued diversity and inclusion of all sorts.[...]I don't want to be at a program where I feel like I have to speak on behalf of all Black people because I'm the only Black person there."(ID 1) "Most programs in general just don't have a lot of residents that are people of color.[Inthatcase, programs needed to] realize [it] was an issue and it was kind of on their frontline for things that they wanted to address."(ID 2) "My co residents or attendings -will call me and say, 'Hey, I have this Spanish-speaking patient, are you in the hospital?Can you help translate?It's amazing.It makes me feel so fulfilled."(ID5)(Continued) open access -michjmed.org