Institutional Context

Dominican University (DU) is a small, Hispanic-Serving Institution in the Chicago suburbs, enrolling 2,136 undergraduates and 1,060 graduate students in the 2022–2023 academic year. The institution has a religious affiliation (Sinsinawa Dominican order of sisters of the Roman Catholic Church) and a strong, historical commitment to a social justice mission. In the 2022–2023 academic year, 53% of all students identified as Latinx/Hispanic, 32% as white, 6% as African American, 3% as Asian, and 5% as two or more races or other groups. The university, which employs 139 full-time faculty and 306 part-time faculty, is organized into four colleges: Arts and Sciences, Health Sciences, Business, and Applied Social Sciences.

Teaching effectiveness is a central component of faculty evaluation for purposes of retention, tenure, and promotion. The university-wide faculty appointments committee reviews all full-time faculty for retention, tenure, or promotion, and the post-tenure review committee reviews tenured faculty. Both committees draw on evidence that typically includes a written narrative, student evaluations of teaching, and letters written by chairs or department colleagues that often reference teaching observations; letters from institutional colleagues are also often submitted, which sometimes include comments on peer observation of teaching. Committee members have often noted that the review materials submitted both by the faculty candidates and their chairs or deans often omit any mention of our official criteria for teaching effectiveness at all, even as they may describe teaching practices at length. The fact that the criteria were seldom mentioned suggested to committee members that they were perceived as not relevant, useful, or meaningful. We clearly needed better criteria that both reflected recent research on teaching and learning and were developed in response to faculty perceptions of what was most important within our shared institutional context if we wanted to strengthen our formal evaluation processes and offer criteria that faculty would draw on to reflect on and improve their personal teaching practices.

Committee members were also concerned about the heavy reliance on student ratings of teaching in our processes, despite the growing body of evidence of bias related to race/ethnicity, gender, discipline, and other factors such as low response rates (Boring, 2017; Fan et al., 2019; Harlow, 2003; Huston, 2006; Merritt, 2008; Smith & Hawkins, 2011). Committee members also recognized troubling patterns in the interpretations of these data, such as chairs or deans focusing on arithmetic means of Likert-like data instead of examining distributions of responses. Optional, written student comments were often cherry-picked by chairs and deans and included in evaluation summaries, to be used as evidence for effective or weak teaching, without consulting numerical data. While we lack formal data of the extent of these issues within our own institution, these problems with student ratings of teaching data are documented across institutions (Hornstein, 2017).

We were eager to revise our processes for evaluating teaching, but we knew that we needed to start by redefining our criteria for effective teaching, which were often vague, not actionable, or completely susceptible to subjective interpretations. For example, our old criteria indicated that effective teaching required “organization of course materials” but didn’t specify the kinds of instructional organization that promote student learning (e.g., aligning classroom objectives to activities). Another criterion purported to measure the instructor’s “ability to communicate subject matter to students,” which seemed both to encourage a transmission-centered mode of instruction and also so vast that it would include nearly every aspect of teaching. Yet another criterion specified “availability to students for academic assistance,” which not only seemed to set a fairly low bar (e.g., just having office hours would seem sufficient) but also did not account for the ways in which implicit bias might shape student perception of faculty availability and approachability. Other criteria mentioned the “regularity with which courses are updated” or “new courses were developed,” which did not apply to those faculty, many times on contingent contracts, who were expected to teach a curriculum developed by full-time faculty in their departments. As a final example, an additional criterion indicated that effective teachers had the “ability to stimulate students intellectually,” which seemed both overly general and difficult to measure as it pointed to something happening within a student’s mind rather than a measurable behavior.

With one exception, these criteria were 20 years old. While there had been a recent addition of “ability to create inclusive classrooms” as a criterion, many faculty did not address this area beyond pointing to statements on course syllabi, perhaps because there were not clear explanations for how this ability might be enacted in specific teaching practices. Furthermore, given our institutional commitment to justice and our deepened understanding of what it means to be a Hispanic-Serving Institution, we realized that our teaching criteria needed to both clarify and center inclusive and equity-minded teaching so that it was not presented as one among many other criteria on a list. We compare our previous criteria to our revised criteria to highlight the deliberate changes we have made (Figure 1).

Figure 1. Revised Definition and Criteria for Effective Teaching, With Previous Criteria as Comparison

Process for Revision

In May 2020, the Provost created a subcommittee composed of faculty representatives from our faculty appointments, post-tenure review, and diversity committees, in response to concerns about the teaching effectiveness criteria. The primary goal was to develop criteria and methods that would not just evaluate teaching, but improve it. However, we also knew that our goal of improvement depended on our ability to craft criteria that responded to our specific institutional and disciplinary contexts and to create a process that established faculty ownership over these criteria, even as we faculty did not always agree on what effective or excellent teaching was. Our entire process, from subcommittee formation to approval of updated criteria through university governance, lasted two years. We note that these two years also corresponded to the first two years of the COVID-19 pandemic in the United States, which surely lengthened our timeline (Figure 2).

Figure 2. Process of Updating Definition and Criteria of Effective Teaching to Approval Through University Governance Process

The first task of the subcommittee was to research and discuss best practices in teaching evaluation. The committee met regularly throughout the summer and fall semesters, gathering research, resources, and identifying best practices from other institutions. While the initial stages of this project were challenging because of the personal, professional, and institutional disruptions caused by COVID-19, we made some tentative conclusions from the literature and other resources. One of the most striking conclusions from our research was how little information we could find on institutional websites. After reaching out for suggestions from the POD listserv, we were encouraged to look at public-facing definitions and processes developed by three research-intensive universities: University of Oregon, University of Southern California, and Vanderbilt University. Two colleagues from the Provost’s office at University of Oregon, Lee Rumbarger, Associate Vice Provost for Teaching Engagement, and Sierra Dawson, Associate Vice Provost for Faculty & Leadership Development, were generous enough to discuss their process of teaching evaluation reform. One of the primary lessons we drew from their experience, which we would translate to our own, was that if we wanted to create a sense of faculty ownership over these teaching criteria, we would need to engage with our faculty colleagues and other stakeholders multiple times, explaining what we were aiming to do and what our current thinking was and asking them for their feedback and suggestions.

Our first step was to draw from our reading of the literature on teaching effectiveness, with much of that literature cited in this article; many subcommittee members shared literature from their disciplines and professional organizations. Then, we reviewed our previous teaching criteria, noting that many concepts were useful, if worded in unhelpful or confusing ways. Then we studied the materials from R1 institutions, including University of Oregon, and invited subcommittee members to gather their insights on effective teaching from their professional experiences, such as inclusive pedagogy training and workshops. For example, many subcommittee members had also been members of other working groups that had worked on core curriculum revisions and had worked in a U.S. Department of Education grant–supported inclusive pedagogy training group. In May 2021, one year after initiation of the project, subcommittee members drafted a definition of effective teaching with new criteria for evaluation of teaching and met with the university provost and deans to receive feedback.

We then entered the broader outreach and feedback stage. We designed a systematic and thorough outreach system, even as we acknowledged that no system could be comprehensive. The subcommittee hosted three open forums during the fall semester, inviting all faculty and administrators. We presented the proposed criteria and gathered feedback. The subcommittee also created a standing survey, with no closing date, so that all faculty and administrators could offer written comments. Subcommittee members also presented within their home colleges and departments to collect feedback. We also contacted key stakeholders directly, such as our Chief Diversity Officer and our Director of Core Curriculum, to gather targeted feedback. Finally, since the subcommittee was formed under the Faculty Appointments Committee, which evaluates faculty for retention, tenure, and promotion, we regularly met with that committee to update the committee and gather feedback. In addition, we also directly consulted with the Post-Tenure Review Committee.

All feedback was organized and sorted by criterion. Subcommittee members met in the early spring 2022 semester to consider all feedback for the purposes of making revisions. The definition and criteria were revised based on comments and feedback, and where we could not incorporate feedback because it conflicted with other suggestions or with the research on learning, we made a note about our reasoning. We felt that responding specifically and thoroughly to our colleagues’ suggestions was essential for establishing a sense of ownership over the criteria. The final, revised definition and criteria were approved by the subcommittee, approved by the Faculty Appointments Committee, and submitted for approval by the overarching governance process in May 2022. The final steps were approval by Faculty Senate and subsequent approval by the overall body of faculty through a series of meetings to consider and vote on the criteria. Prior to the final faculty body votes in the fall of 2022, subcommittee members hosted an opening year workshop on the new criteria and how to use them as a formative system and way to evaluate teaching development.

The criteria and definition were given final approval by the faculty body, with close to 90% approval, in November 2022 and became part of the faculty handbook in the subsequent academic year.

While one of our initial goals was to update our teaching evaluation system, including criteria and evidence, we ultimately fell short of the goal of updating evidence, which is discussed later in future steps and limitations. For more information about our process of reform, please see the appendices for example meeting agendas, survey text, and responses to feedback.

Updated Teaching Effectiveness Criteria

One shortcoming of our previous criteria was that they were presented as a long list, without indication about how they connected to each other or which was most important (Figure 1). With that in mind, we took inspiration from University of Southern California and University of Oregon and organized our new definition and criteria for evaluation of teaching around four main standards. Quality teaching should be inclusive and equity minded, purposeful, both rigorous and supportive, and reflective (Figure 1). We also crafted a preamble to the criteria that provides a definition of teaching excellence that explicitly aligns with our university mission and social justice identity and that grounds our teaching in our institutional status as a Hispanic-Serving Institution. To be clear, while our Hispanic-Serving identity necessarily informs our goal to better articulate what inclusive teaching means for us, we also believe that inclusive teaching is a necessary part of quality teaching at all institutions.

For each of the four main areas and various subcriteria, we made each point clear and actionable and tried to reduce the subjective interpretation of each concept. The process of crafting each concept was largely dialectical, as we first drew on findings from research on learning to express criteria that were based on evidence of how students learn, and then we considered what those findings meant for teaching at our institution, with our specific faculty body, curricular structures, and mission. Below, we describe our new criteria and indicate the sources that informed our thinking.

Given our mission and status as a Hispanic-Serving Institution, we knew that inclusive and equity-minded teaching needed to come first. As we define it in our new evidence-based criteria, an inclusive and equity-minded instructor draws on students’ identities and backgrounds as assets in learning (Banks, 2008; Pardon et al., 2002; Salazar et al., 2010), uses anti-oppressive pedagogies (St. Clair & Kishimoto, 2010), adopts universal design for learning recommendations (Burgstahler & Cory, 2008; Rose et al., 2006; Thurber & Bandy, 2018), recognizes and navigates power dynamics in the classroom (Burns, 2014; S. R. Harper & Davis III, 2016; Sue & Spanierman, 2020), and draws on evidence to address equity gaps (Lovett & Hershock, 2020).

Two of the criteria areas, “purposeful” and “rigorous and supportive,” are fairly straightforward and generated little feedback or faculty dissent. While most of the subcriteria under these two areas would likely be generated by faculty at any institution as obvious elements of effective teaching, it’s worth stating that these were not intentionally stated as goals in our previous teaching criteria. We defined purposeful teaching as establishing clear and meaningful learning goals for the course (Fink, 2013; Kezar & Maxey, 2016), providing well-organized course materials (Wieman, 2015), offering formative and summative assessments that are aligned with the course goals (Wiggins & McTighe, 1998), preparing thoughtful class plans (and adjusting them appropriately in response to student needs) (Bransford et al., 2000; Hake, 1998), interacting with students consistently and respectfully (Greer, 2014), and strategically using a variety of teaching and learning methods (Brame, 2016).

We replaced a previous criterion about being available to students with more comprehensive and balanced criteria related to rigorous and supportive teaching, which we define as setting challenging but reasonable expectations for all students (Kezar & Maxey, 2016; Lundberg et al., 2018; Margolis & McCabe, 2006; Orey, 2010; Yarborough & Fedesco, 2020); offering students options for receiving support, whether from university services or other sources; giving students useful and timely feedback (Brinko, 1993; Davis, 2009; Dohrer, 1991); communicating effectively with students via multiple modes; promoting a growth mindset for all involved in learning (Dweck, 2006); providing a consistent and transparent grading system (Galina, 2016; Walvoord & Anderson, 2010); and applying policies consistently, fairly, and equitably (Close, 2009; Whitley et al., 2000).

The fourth area, reflective teaching, was also not represented in our previous criteria, even though our university mission emphasizes contemplation, and our faculty colleagues almost uniformly recognize this area as vital to our teaching. For us, reflective teaching involves drawing on feedback, data, evidence, and current findings of pedagogical scholarship for further improvement of teaching and learning (Brookfield, 2017; J. Harper & Kezar, 2023; Lewis, 2001; Lovett & Hershock, 2020; Salazar et al., 2010); regularly updating courses based on advancements in one’s disciplines; fostering student research and inquiry; trying new approaches and learning from failures; and recognizing the collaborative aspect of teaching (Gillman et al., 2016; National Association of Biology Teachers, 2012; Shulman, 1993). We wanted to elevate collaboration in our criteria to recognize that while efforts such as reflecting on learning across sections of the same course or engaging in curricular conversations, revisions, and assessments are time consuming and potentially formative, they are often not included in evaluations of teaching. Furthermore, explicitly valuing collaboration helps shift our perspective of teaching from a private practice to a more learning-centered view of teaching as a shared endeavor (Gillman et al., 2016).

General Challenges Encountered in the Process of Reform

In our process of revising our teaching evaluation criteria, we encountered some issues that could be present at any institution. We describe these issues below along with our responses.

The Goldilocks problem. During our process of research, drafting, feedback, and revisions, several issues arose. One issue was the Goldilocks problem: too many criteria, too specific, not enough criteria, missing concepts. Many of our colleagues commented that our new criteria contained too many items and that instructors would be unreasonably burdened to demonstrate achievement of all of them. Our criteria had grown, with the previous list of criteria listing 10 conceptual items, whereas our new criteria contain 23 subcriteria organized into four areas. At the same time, we also received streams of feedback pointing to a key concept perceived to be missing from our criteria, e.g., that we should specifically include culturally relevant pedagogy in our inclusive and equity-minded area. Both points were legitimate, and we had to navigate the tension between being actionable and comprehensive. Ultimately, we determined that our previous criteria were so broad and vague that one conceptual item in fact included many hidden subcriteria. For example, the criterion of “able to stimulate students intellectually” would include the choice of instructional materials, the ability to design meaningful assessments, and the ability to keep students engaged in class through a variety of pedagogical approaches.

Our new criteria were drafted to be specific enough for our colleagues to readily demonstrate how they meet or are working toward meeting the criterion. Another solution to this problem is to simultaneously address the criteria of the system and evidence used to evaluate teaching, such that they are aligned and actionable. However, in our process it became clear that we could not make timely progress on reform of evidence, so we had to develop better solutions to make the new criteria acceptable.

We worked through various alternative systems, which ranged from requirements to meet all criteria, to use of structured reflection inventories as supports for faculty under review, to systems where faculty chose a subset of criteria to highlight for any given review. We arrived at a solution where faculty under review would demonstrate how they meet all four areas, through two to three subcriteria of their choosing under each area. In the late stages of governance approval, this requirement was included in the text of the faculty handbook as protection and clarity for faculty under review. This is a workable compromise and allows forward momentum toward reform. We also note that this specific requirement can be changed without altering the overall structure of the criteria and future assessment is a necessary part of the long-term process of reform.

Excellence versus effectiveness. An interesting problem that arose during our process of drafting and feedback was the issue of framing our teaching goals as achieving excellence versus effectiveness. In our initial drafts, we stated that the goal of the criteria was excellent teaching, whereas our later criteria were stated in terms of effective teaching. While some faculty and administrators favored the aspirational goal of excellence, others were concerned that excellence could require too much subjectivity on the part of evaluators or set impossibly high standards. The subcommittee concluded that although excellence is a worthy goal of teaching, phrasing teaching as effective would be more helpful to faculty and would allow the process of transition to a new system to proceed more smoothly.

Future Steps and Limitations

Here we have described our process of reform of our system of teaching evaluation. Although we set out to reform our total system, including criteria and evidence, we fell short in making meaningful change to the evidence that should be used to analyze each criterion. This was especially the case with the role of student ratings in the overall process. While it’s true that any evidence of teaching would be subject to various types of bias and limitations, there are better solutions to making important decisions about people’s professional trajectories than drawing primarily on student ratings. For example, there are known and egregious issues with student ratings of teaching data, yet our university and many other institutions over-rely on these data to evaluate teaching (Deslauriers et al., 2019). The next steps at our institutions and others are to acknowledge the body of literature on limitations and best practices related to evaluation of teaching and to make meaningful reforms. While student ratings provide some helpful evidence, they must be considered a part of and not the entirety of the evidence.

Because our criteria are grounded in evidence of learning, we believe that they will not only increase our colleagues’ confidence in them for purposes of formal reviews but also be aspirations and anchors for self-reflection and faculty development in departments, programs, and colleges. For example, we can imagine our teaching center offering a series of workshops on universal design for learning, growth mindset, anti-racist teaching, or using pedagogical literature in one’s field in ways that align and support the new criteria. Finally, the new structure may allow a better process for revision. Over time, some subcriteria may be difficult to enact or evaluate or may be more or less useful than others. With the new structure of four areas, subcriteria could be added, removed, or revised, while maintaining the overall structure of the system.

An interesting inspiration from our colleagues at University of Oregon was their use of their updated teaching system for teaching awards. Although we are a teaching-focused institution, we only have two awards given annually to the ˜140 full-time faculty and none to part-time faculty. Our new teaching criteria can be used as the basis for an expanded set of teaching awards to recognize skillful teaching in an objectively decided manner.

Discussion and Conclusion

Here we have described a revised set of teaching evaluation criteria and our process of change in our context as a small university that is also minority serving. According to the National Center for Education Statistics, 80% of higher education institutions in the United States have fewer than 5,000 students (National Center for Education Statistics, 2023). Our work on this process would have been impossible without the literature on reforming approaches to teaching evaluation. We also acknowledge that public-facing resources from multiple institutions and organizations were invaluable in our process, especially from University of Oregon, University of Southern California, Vanderbilt University, University of Colorado, and University of Kansas. Each of those listed institutions is an R1, research-intensive, university, and there is notable absence of resources from primarily undergraduate institutions (PUIs) and small, liberal arts and sciences institutions.

While quality teaching probably has many commonalities across institutions, there is a sparse literature on teaching quality at smaller institutions. Mooney and Reder (2008) wrote about the irony of the stated importance of quality teaching at smaller institutions, but where teaching goals are not often defined. The process of earning tenure at research institutions is often described as navigating hidden requirements for scholarships and grants (Cate et al., 2022); at many teaching-focused institutions, expectations for teaching are also often hidden. Our own teaching effectiveness criteria were only accessible in our faculty handbook, which is not publicly accessible, so that faculty candidates cannot easily know what the teaching expectations are. In a search for teaching criteria from peer institutions, we found that they were not publicly accessible and were not shared after direct requests, or we were told that such criteria did not exist yet. Prospective students and their families are also interested in understanding how their potential institutions value and promote good teaching (McMurtrie, 2023).

As a necessary correction, we are committed to sharing our resources and description of our processes. Our goal is to include our criteria and, eventually, updated evidence on our public-facing university website, not just contained in our publicly inaccessible faculty handbook. Our current teaching criteria are most likely inaccessible to our part-time faculty, and more access to criteria as teaching goals can allow us to include part-time faculty in teaching development.

We hope that our work is a step toward making teaching goals visible and comparable across institutions. Interestingly, Australia has a de facto set of national teaching effectiveness expectations for higher education (Devlin & Samarawickrema, 2010). In the absence of national standards of effective teaching in the United States, a solution is to better define effective teaching clearly within institutions.

Our revised teaching criteria and our description of the process of reform can help faculty and staff at other teaching-focused institutions. A central tension at many teaching-focused institutions is that teaching loads are high, there are few support staff, there is high turnover in staff and administrators, but effective teaching is central in decisions about faculty careers (King-Smith et al., 2021; Kortegast & Hamrick, 2009). The net impact of these issues is that faculty do not have consistent guidance for becoming better teachers and the work burden of development often falls on individual faculty. Clear and transparent teaching criteria can help faculty, staff, and administrators communicate about goals of classroom work, even in unstable environments.

Another attribute of many smaller institutions, especially minority-serving institutions, is that the student body has more traditionally underrepresented students and more students from less-wealthy families (Fry & Cilluffo, 2019). There are persistent gaps in outcomes for many of these students. For example, Latine students have a meaningful gap in college completion rates compared to white students (Excelencia in Education, 2024; Mora, 2022). There is a rich literature on teaching practices and behaviors that improve student outcomes, especially for underrepresented students, but as stated earlier, these skills and behaviors should be explicitly stated as instructional goals (Tanner, 2013). Our approach to this issue was to expand our goal of inclusive and equity-minded teaching into many detailed subcriteria, which were clear and measurable. We also added aspects of inclusive pedagogy throughout our teaching criteria. In a notable report on the findings of the College Educational Quality study, Campbell (2023) stated that while overall teaching quality at some liberal arts colleges was relatively high, inclusive pedagogy was an overall weakness, pointing to the need for better inclusive and equity-minded teaching at smaller institutions.

Finally, it’s important to acknowledge the complexity of generalizing about higher education institutions. The trends we have stated do not apply to all teaching-focused institutions, where some liberal arts institutions are highly resourced (Campbell, 2023). Additionally, many larger, regional institutions share the traits we have noted, and there are several minority-serving institutions that are large, research-intensive institutions (Campbell, 2023; Martinez & Garcia, 2020).

An important next step is for more higher education institutions to craft and share their criteria for effective teaching. Additionally, students and institutions themselves would benefit from more evidence of how we are meeting our goals of quality teaching. Campbell (2023) noted the general lack of information on teaching quality for higher education institutions in the United States; one finding from the College Educational Quality study is that there is a general mismatch between faculty self-perceptions of teaching quality and more objectively determined quality. While the success of reforms of teaching evaluation remains an open question, hopefully more transparency within institutions about teaching goals and increased sharing of information between institutions can improve teaching and learning.

Conflict of Interest Statement

The authors have no conflicts of interest.