This was a descriptive, mixed methods investigation involving the distribution of a survey and the subsequent analysis of quantitative and qualitative data. Quantitative numerical data were obtained through Likert scale survey items and qualitative data through respondents’ narrative comments (see Appendix A). This design was utilized to identify the factors that predict the use of inclusive teaching practices and to identify instructors’ perspectives of both the barriers to implementation and promising change initiatives that are occurring. This study involved thematic analysis and not content analysis. These research methods have overlap but also differences. As described by Neuendorf (2019), thematic and content analyses each involve coding data and identifying variables or constructs. Content analysis comes from a positivist approach to use quantitative techniques and thematic analysis from a more constructivist approach. In thematic analysis, the instructors’ comments are considered the data, and the codes develop as themes emerge. Content analysis assumes that the comments are the phenomena to be studied. In content analysis, the coded variables are typically summarized in a quantitative manner (Neuendorf, 2019). For thematic analysis, as noted by Neuendorf, “the frequency of occurrence of specific codes or themes is usually not the main goal of the analysis” (p. 212).

The Inclusive Teaching Questionnaire was designed for this study and consisted of 18 Likert scale items, eight open-ended questions, and an array of demographic questions. The full instrument is available in Appendix A. Responses to Likert scale items, which measured agreement on a 6-point scale (1 = strongly disagree to 6 = strongly agree) were analyzed in SPSS. To address different aspects of inclusive teaching, the Likert scale items were presented in five sections: knowledge of inclusive teaching (four items), utilization of inclusive teaching practices (four items), departmental support of inclusive teaching (three items), institutional support for inclusive teaching (three items), and opinions about the progress being made at various levels in regard to inclusive pedagogy (four items).

To provide content validity evidence, the items on the questionnaire were generated by content experts in faculty development, educational psychology, and inclusive teaching practitioners (AERA, APA, and NCME, 2014). The five sections of the survey each had the following reliability measures: knowledge of inclusive teaching (α = 0.94), utilization of inclusive teaching practices (α = 0.84), departmental support of inclusive teaching (α = 0.83), institutional support for inclusive teaching (α = 0.82), and opinions about the progress being made at various levels in regard to inclusive pedagogy (α = 0.80).

The survey questions were distributed anonymously using Qualtrics. Invitations to participate were directly sent to over 30 higher education institutions (generally to their CTLs for subsequent distribution to their faculty) that are located in various geographical regions of the United States and represent diverse institutional types, including private and public, large and small, universities, state colleges, liberal arts colleges, historically Black universities and colleges, seminaries, and community colleges. To further the reach of the survey, invitations to participate were also sent through consortiums and listservs (e.g., Professional and Organizational Development [POD] Network, Consortium for Faculty Diversity [CFD], Council on Undergraduate Research [CUR], New England Faculty Development Consortium [NEFDC], etc.) and posted on social media platforms (e.g., Facebook, LinkedIn). The responses were collected from January 12, 2019, to February 26, 2019. The median response time was 14 minutes, 55 seconds. Respondents did not receive compensation for participating in the study, and information related to the purpose, procedures, institutional review board approval, benefits, risks, and other relevant information was provided prior to participation. The survey was closed when the responses slowed down to nearly zero.

A total of 566 participants started the survey. Of that number, 306 participants reached the end of the survey and submitted responses. In total, 214 participants responded to all of the questions that were included in the quantitative analysis portion of this study. Given that the survey distribution included networks and social media, response rates could not be calculated. The sample included 180 females (84.11%), 174 individuals that identified as White and non-Hispanic (81.31%), 118 tenured or tenure-track faculty (55.14%), 175 full-time faculty (81.78%), 97 faculty who worked at a doctoral granting institution (45.33%), 162 faculty who had participated in professional development activities that focused on inclusive teaching (75.70%), and 132 faculty from STEM fields (61.67%). Faculty from 44 academic disciplines responded to the survey. Disciplines were classified as STEM or non-STEM fields based on the U.S. Department of Homeland Security, Immigration and Custom Enforcement criteria (Department of Homeland Security, 2016). These demographic breakdowns, geographic location of respondents, and the mean number of years of teaching experience (14.36 years; SD = 8.94) can be found in Appendix B.

The mean sum scale score (i.e., we added scores for all items in the scale, divided by the number of items, then calculated the mean across all participants) across questions measuring the respondents’ perceived knowledge of inclusive teaching and inclusive teaching practices was 19.56 (SD = 4.44, n = 214; Figure C1). This mean fell between the expected values representing somewhat agree and agree, at 16.00 and 20.00, respectively. The highest mean (20.35, SD = 3.87, n = 162) was found for those who had participated in professional development addressing inclusive pedagogy, and the lowest mean (17.12, SD = 5.19, n = 52) for individuals stating that they did not participate in professional development addressing inclusive pedagogy.

The mean sum scale score across questions measuring the utilization of inclusive pedagogies was 20.43 (SD = 2.89, n = 214; Figure C2). This mean fell between the expected values representing agree and strongly agree, at 20.00 and 24.00, respectively. The lowest mean, 19.83 (SD = 3.00, n = 52), was found for the group of respondents indicating that they did not participate in professional development addressing inclusive pedagogy, although this value was still in the range of somewhat agree to agree responses. The highest mean, 21.18 (SD = 3.12, n = 82), was present in faculty respondents from non-STEM disciplines.

The mean of the summed scores of all respondents on questions measuring departmental support for inclusive teaching was 9.10 (SD = 4.00, n = 214; Figure C3). This score fell between the expected values for somewhat disagree and somewhat agree, 9.00 and 12.00, respectively, but closer to somewhat disagree. The lowest mean was found in respondents who indicated they did not participate in professional development addressing inclusive pedagogy, at 7.85 (SD = 3.89, n = 52), which fell between the expected value of 6.00 for disagree and 9.00 for somewhat disagree responses. Additionally, it is noteworthy that faculty who identified as being either tenured or on the tenure track had a mean of 8.44 (SD = 4.05, n =118), whereas the highest mean in all groups, 9.91 (SD = 3.82, n = 96), was found in those identifying as non-tenure track, though this value is still below the mean expectation for somewhat agree responses.

The overall mean for the summed questions addressing institutional support for inclusive teaching was higher than that seen for departmental support, at 11.97 (SD = 3.58, n = 214), almost precisely equaling the expected value for somewhat agree responses of 12.00 (Figure C4). The highest mean value again was identified in the non-tenure track respondents, at 12.66 (SD = 3.26, n = 96). The lowest mean was found from respondents indicating they did not participate in professional development addressing inclusive pedagogy, at 9.85 (SD = 3.99, n = 52), closest to the somewhat disagree expected value of 9.00.

A simultaneous multiple regression equation was used to examine variables that predicted utilization of inclusive teaching practices. The assumptions of linearity, independence, normality, and homoscedasticity were tested and met. The dependent variable was the sum scale score of the questions measuring utilization of inclusive teaching practices. Independent variables included the sum scale score of the questions measuring knowledge of inclusive teaching; the sum scale score of the questions measuring departmental support of inclusive teaching; the sum scale score of the questions measuring institutional support for inclusive teaching; years of teaching experience; participation in professional development addressing inclusive pedagogy (no = 0, yes = 1); gender (male = 0, female = 1); ethnic or racial minority (non-White or Hispanic = 0, White and non-Hispanic = 1); tenure track status (non-tenure-track = 0; tenured or tenure-track = 1); discipline (non-STEM = 0, STEM = 1); and teaching at a doctoral granting institution (no = 0, yes = 1).

The results of the multiple regression analysis measuring predictors of utilization of inclusive teaching approaches indicated that the 10 predictors explained 29.10% of the variance (R² = 0.29, adjusted R² = 0.26, F(10, 203) = 8.33, p < .01). The results indicated that discipline (i.e., STEM versus non-STEM; β = −1.17, t(202) = −3.21; p = .002) and the knowledge score (β = 0.30, t(202) = 7.11; p < .001) significantly predicted the utilization score (Table 1). Using this model, the average STEM faculty had an average utilization score 1.17 points less than non-STEM faculty, and faculty with higher knowledge scores had higher utilization scores when holding all of the other variables constant.

The Inclusive Teaching Questionnaire asked respondents about their perceptions of the level of progress being made on inclusive teaching efforts at four distinct levels: individual, departmental, institutional, and higher education overall. Descriptive statistics and a logistic regression equation are presented for each level (individual, departmental, institutional, and across higher education). The regression equations examined how individuals with different demographics perceived the progress that is made at each level. The dependent variable for each equation was a single item rating agreement for a specific level. Categories for agreement were collapsed into two categories (disagree = 0, agree = 1) because the sample size in each of the original six categories of agreement was too small when cross-tabbing across multiple demographic variables. The independent variables included participating in professional development addressing inclusive pedagogy (no = 0, yes = 1); gender (male = 0, female = 1); race/ethnicity (non-White or Hispanic = 0, White and non-Hispanic = 1); tenure track status (non-tenure track = 0; tenured or tenure-track = 1); discipline (non-STEM = 0, STEM = 1); and teaching at a doctoral granting institution (no = 0, yes = 1).

Descriptive statistics for the item “I am making enough progress with regard to inclusive pedagogy” are broken down by demographic characteristics in Figure C5. The overall mean of all respondents was 4.33 (SD = 1.16, n = 214), a value that fell between somewhat agree (4.00) and agree (5.00). The demographic with the highest mean was those who had participated in professional development addressing inclusive pedagogy at 4.52 (SD = 0.97, n =162), and the lowest mean was in the group who stated they did not participate in professional development addressing inclusive pedagogy at 3.71 (SD = 1.46, n = 52). This represented the only group falling below the somewhat agree threshold. Results from the logistical regression equation related to individual progress are presented in Table 2. Variance inflation factor and the Box-Tidwell test were examined to ensure that the assumptions of multicollinearity and the linearity of independent variables with log odds were met, respectively. The logistic regression model was statistically significant, χ2(7) = 28.35, p < .001. The model explained 20.10% (Nagelkerke R²) of the variance in agreement and correctly classified 81.3% of cases. Using this model, individuals who participated in professional development activities were 6.25 times more likely to agree with the statement that they were making enough progress with inclusive pedagogy.

Descriptive statistics for the item “My department is making sufficient progress with regard to inclusive pedagogy” are broken down by demographic characteristics in Figure C6. The overall mean of all respondents was 3.21 (SD = 1.29, n = 214), closest to the value for somewhat disagree (3.00). The demographic with the lowest mean (2.92, SD = 1.34, n = 52) was those who did not participate in professional development addressing inclusive pedagogy, with the next lowest mean from males (2.97, SD = 1.19, n = 34). The highest mean was found in the non-tenure track instructors, at 3.45 (SD = 1.26, n = 96), though this value still fell below that of somewhat agree (4.00). The logistic regression model was not statistically significant, χ2(7) = 8.13, p = .32. The model explained 5.00% (Nagelkerke R²) of the variance in agreement and correctly classified 53.30% of cases (Table 3). Using this model, non-tenure track faculty were 0.52 times more likely than those that identified as tenured or on the tenure track to agree with the statement that their department is making sufficient progress with inclusive pedagogy.

Descriptive statistics for the item “My institution is making sufficient progress with regard to inclusive pedagogy” are broken down by demographic characteristics in Figure C7. The overall mean of all respondents was 3.36 (SD = 1.26, n = 214), closest to the value for somewhat disagree (3.00). The demographic with the lowest mean was those in non-STEM fields (3.07, SD = 1.33, n = 82), while the demographic with the highest mean was those in STEM fields (3.55, SD = 1.19, n = 132). All demographic groups responding to this question had means that were between somewhat disagree (3.00) and somewhat agree (4.00) responses. The logistic regression model was not statistically significant, χ2(7) = 4.66, p = .74. The model explained 2.70% (Nagelkerke R²) of the variance in agreement and correctly classified 53.70% of cases (Table 4).

Descriptive statistics for the item “Higher education is making sufficient progress with regard to inclusive pedagogy” are broken down by demographic characteristics in Figure C8. The overall mean from all respondents was 3.18 (SD = 1.13, n = 214), closest to the somewhat disagree value (3.00) and the lowest overall mean from any of the four levels of query regarding progress in inclusive teaching. The lowest means were found in those that did not participate in professional development addressing inclusive pedagogy (2.98, SD = 1.08, n = 52) and those in non-STEM fields (2.98, SD = 1.09, n = 82). The highest mean was found in those in STEM fields at 3.31 (SD = 1.15, n =130). The logistic regression model was not statistically significant, χ2(7) = 13.63, p = .06. The model explained 8.4% (Nagelkerke R²) of the variance in agreement and correctly classified 60.4% of cases. Within this model, STEM faculty were 2.17 times more likely than non-STEM faculty to agree with the statement that higher education is making sufficient progress with regard to inclusive pedagogy (Table 5).

A thematic analysis of responses to the open-ended question “Describe the largest barrier to the success of inclusive pedagogy at your institution” revealed a variety of obstacles to inclusive teaching efforts, as identified in Appendix D. Barriers were either personal or institutional. Personal barriers included lack of awareness, fear, unwillingness to change, not feeling responsible, and challenges with promoting inclusion in student interactions with one another. Institutional barriers included lack of administrative support, inadequate resources and lack of incentives, lack of diversity and systemic “isms,” and limited opportunities for discussion. Representative comments are also provided from the 199 comments.

When analyzing participants’ answers to the open-ended question “Which types of initiatives have the potential to advance inclusive pedagogy at institutions of higher education?” several themes emerged. Common themes are described in Appendix E. Initiatives reflected institutional commitment and resources (administrative and faculty buy-in, more professional development opportunities addressing inclusive teaching, more resources and incentives, and more opportunities for discussion), departmental supports (general initiatives and better recruitment and retention of minoritized faculty), data-driven approaches (usage of data analytics and conducting assessments that measure impact), and student supports both inside and outside of the classroom.

The respondents in the current national study on average reported being knowledgeable about inclusive teaching. Many indicated that they utilized inclusive practices in their courses. Knowledge of inclusive teaching was a key predictor as to whether respondents reported implementing such approaches. While individuals who engaged in professional development activities were also more likely to agree with the statement that they were making enough individual progress with inclusive teaching, reported participation in professional development was not a key predictor of utilization of inclusive teaching approaches.

One reason that participation in professional development may not have been a key predictor of utilization is that respondents may have gained awareness of inclusive teaching approaches through experiences outside of formal workshops or initiatives, such as their own personal experiences in the classroom (as a student or teacher) and more informal conversations with colleagues. However, respondents still expressed value in having professional development experiences addressing inclusive teaching as is evident in the thematic analyses. Furthermore, the difference in utilization scores between those participating in professional development and not doing so is a narrow range for this sample (not statistically significant). These findings corroborate the literature on the critical importance of teaching supports in the implementation of inclusive teaching (Bathgate et al., 2019; Brownell & Tanner, 2012) and highlight a key role for educational development.

In addition to participation in educational development for inclusive teaching, being from a non-STEM discipline was also a key factor for reported implementation of such approaches in this particular sample. STEM courses can face a variety of perceived structural challenges—such as larger class sizes in gateway courses, heavy content where prior knowledge can play a key role in achievement, more challenges or questions around how to integrate diversity and inclusion within the curriculum, and instructors teaching the way they have been taught—that can result in instructors implementing less inclusive approaches (Brownell & Tanner, 2012; Henderson & Dancy, 2007; Herreid & Schiller, 2013; Michael, 2007). The ongoing focus on inclusive teaching in STEM courses in the literature supports this finding, which highlights that the implementation of inclusive teaching poses a distinct perceived and/or actual challenge for instructors within such disciplines (Dewsbury & Brame, 2019; Friedrich et al., 2017; Killpack & Melón, 2016; Penner, 2018; Tanner, 2013).

Respondents also reported more support from their institutions rather than their departments on inclusive teaching efforts. They generally did not perceive that their departments, institutions in general, or higher education were making significant progress with regard to inclusive pedagogy. We hypothesize that the higher departmental and institutional support means found for non-tenure track instructors, while not statistically significant, were reflective of the large number of non-tenure line respondents in this sample being from fields such as education and psychology, where inclusion has a historical emphasis in the discipline. Furthermore, this outcome may also be a result of tenure-track faculty perceiving less support of their teaching efforts due to the heavy emphasis on research at institutions where scholarship production is critical for tenure and promotion. Future investigations are necessary to better understand such findings.

Respondents reported a variety of barriers to the implementation of inclusive teaching at their institutions, such as lack of awareness that inclusion was an issue or lack of professional development; lack of buy-in from their colleagues and administration; fear; not wanting to take on the responsibility; not having enough time or proper resources; lack of incentives; an unwillingness of their colleagues to change their teaching practices; and need for more discussions, assessment and data measures, and a variety of student supports. Change initiatives for promoting inclusive teaching included more and sustained professional development, resources, incentives and rewards, administrative support, student support, equitable hiring and support for diverse faculty, department-level initiatives, data on student diversity and formal assessment plans, and more discussions. Such initiatives are consistent with organizational change theories around leading change, involving having a shared vision around the change and the buy-in of various campus partners (e.g., students, faculty, administrators) to enact policies and procedures and provide resources to further reform efforts (Kezar, 2013). These open-ended responses triangulate the survey findings with reference to perceived barriers to inclusive teaching efforts and initiatives perceived to lead change.

There are some seeming contradictions in the data that could be explored in future work. For example, participating in professional development addressing inclusive teaching was not a statistically significant predictor for the utilization of inclusive teaching practices, but participants also described wanting more professional development to advance inclusive teaching efforts. As mentioned previously, the majority of instructors taking the survey were utilizing inclusive teaching approaches and may have already been invested in using such approaches even without formal professional development yet still recognized the importance and value of professional development that addresses inclusive teaching. Another factor is the degree to which, with regard to quality as well as frequency, the professional development experiences in which the respondents participated led to their adoption of inclusive teaching strategies. Sustained professional development experiences are known to have higher efficacy than single workshops or events (Derting et al., 2016). In other words, participating in one or few inclusive teaching workshops might not be sufficient to lead to adoption of such teaching practices. Furthermore, if the events were not designed to facilitate a change in practice, utilization of inclusive teaching strategies may not be an outcome.

While this survey appears to be a promising tool for assessing different perceptions of inclusive teaching, more validity evidence ultimately needs to be gathered. The survey was constructed in consultation with a panel of experts to provide some validity evidence related to test content (AERA, 2014) and ensure that the items accurately represented inclusive teaching. Nevertheless, examining how responses correspond to observations of actual implementation of inclusive teaching and correlating responses to other measures of teaching practice or campus climate would provide additional validity evidence in relation to other variables (AERA, 2014) and provide insight into how this survey may be improved. Operationalizing terms like sufficient progress in queries about advancements related to inclusive teaching could ensure more consistency in responses across participants. The survey was developed for the purpose of conducting an exploratory national research study. Adapting the survey for other purposes (e.g., evaluating departmental or campus climate or diagnosing the implementation of inclusive teaching by individual faculty) would require researchers to gather additional validity evidence to support their interpretation of the data.

The survey relies on self-reported data and was distributed via a variety of channels, and the recruited sample therefore may not be representative of the pedagogical practices and opinions of faculty members across higher education. Furthermore, the survey was voluntary; thus, those that were more accepting or knowledgeable of inclusive teaching may have been more eager to complete this survey. The survey may have been taken by multiple individuals at the same institution. Additionally, attrition during the survey may have impacted the results as similar individual characteristics may have impacted participants’ willingness to spend more time answering all the quantitative survey questions and providing in-depth responses to the qualitative survey questions. While the final response allowed us to conduct the analyses presented, a larger number of responses could further clarify relationships found within the data analysis (e.g., implementation by faculty in STEM versus non-STEM fields).

The outcomes of this study highlight the critical importance of educational development that focuses on inclusive teaching. In particular, for this sample, the findings reveal that knowledge development in inclusive teaching practices was a significant predictor of whether instructors implement such approaches and that participation in professional development focused on inclusive teaching was not. While further study is needed, such outcomes suggest that professional development on inclusive teaching should be meaningfully designed to support adoption.

Institutions could leverage existing programming and relationships when developing additional training related to inclusive teaching. When asked “Which departments or offices (if any) provide professional development for inclusive teaching at your institution?” respondents commonly noted offices such as CTLs and offices of diversity and inclusion. Less regularly mentioned were offices of institutional or education effectiveness, grant-funded programs, and specific departments. By understanding how faculty receive advice and support on their teaching within an institution, educational developers may be able to increase faculty buy-in and adoption. Educational developers may also find it beneficial to emphasize intensive institutes, faculty learning communities, and other forms of more sustained professional development that support the knowledge development of instructors and monitor or assess such learning in place of focusing on holding one-time events that may or may not be as impactful at building foundational understanding of inclusive teaching approaches. An example initiative that provides sustained support on inclusive teaching and other evidence-based practices is the Summer Institutes on Scientific Teaching. These multi-day intensive institutes provide tools and a supportive community to help instructors implement evidence-based teaching approaches. Participants have self-reported using inclusive teaching approaches at higher levels within the two years after the summer institute (Pfund et al., 2009).

Given that being from a non-STEM discipline was a significant predictor of adopting inclusive teaching approaches, a promising change initiative is providing directed professional development activities that support STEM faculty who face actual or perceived challenges with implementation. An initial focus on implementing smaller-scale methods in gateway courses that often emphasize heavy content may be more palatable and impactful in supporting student persistence (Seymour & Hunter, 2019). Passing gateway courses is known to be important for student persistence, and recent work has found that introductory chemistry courses were particularly critical for underrepresented students (Harris et al., 2020). Furthermore, research suggests that physics, computer science, and engineering attract even lower-achieving males, whereas females are less likely to pursue those disciplines unless they have high achievement backgrounds, which can account for gender gaps (Cimpian et al., 2020). Inclusive teaching approaches in such foundational courses may include ensuring accessibility of newly developed course materials, integrating diverse examples, creating an environment that promotes equitable student participation, implementing brief active learning exercises such as think-pair-share, among others. Educational developers, such as professionals who perform work in institutional CTLs, can be strong advocates of equitable and inclusive teaching, demonstrating their values through programming, services, and emphases within consultations. They can play a role in coordinating discussions around equity and inclusion in classroom teaching. Additionally, educational developers can strategically offer services at the department level around inclusive teaching. These may involve specific learning communities geared toward equitable and inclusive teaching for faculty within departments or faculty groups who teach multi-section courses. Educational developers may also provide forums for departments to discuss their largest challenges around equity and inclusion and develop actionable items to address such concerns. Through their initiatives, educational developers can advocate for multicultural frameworks, over those that are colorblind, as evidence suggests that such approaches are associated with the implementation of inclusive teaching (Aragón et al., 2017).

Data-driven approaches uncovered in the thematic analyses may potentially be useful at raising the awareness of instructors in quantitative fields such as STEM disciplines but also have potential in non-STEM disciplines. Such initiatives may include highlighting statistics within the literature and, if available, within the institution that reveal the exclusion of particular student groups or other inequities. Increasing the level of awareness of achievement gaps between student groups is a promising avenue by which to foster change. Such analytics can also be calculated at the departmental, program, or multi-section courses level. Educational developers can work with individual faculty members as well as departments to help measure the impact of implementing inclusive teaching approaches. In general, understanding what really matters to instructors implementing inclusive teaching strategies can support institutional change (Kezar, 2013). The results of this study are also particularly informative for inclusive teaching initiatives addressing inequities revealed through the COVID-19 pandemic and racial unrest occurring across the nation.