Institutional Context

Massachusetts Institute of Technology (MIT) is a private, STEM-focused research university with more than 2000 faculty and teaching staff, 1500 postdocs, 7300 graduate students, and 4500 undergraduate students. In 2024–2025,¹ women accounted for 48% of the undergraduate and 42% of the graduate student population. Fifty-seven percent of undergraduates and 21% of graduate students self-identified as members of one or more US minority groups. International students accounted for 12% of the undergraduate and 40% of the graduate student population. In the undergraduate class of 2028, 24% of students were Pell Grant eligible and 20% were the first generation in their family to attend college.

Across MIT, there are 30 academic departments across five schools and one college. MIT is a decentralized institution, with much of the decision-making authority occurring at the departmental level, as illustrated in recent university-wide strategic action plans that emphasize local plans at the department level. Our center for teaching and learning (the Teaching + Learning Lab) is a centralized resource for teaching support at the institution with staff who are experts in teaching, learning, research, and evaluation.

At MIT, instructors have significant flexibility on the format and content of their syllabi, except for certain policies pertaining to exam timing, supplemental class scheduling, assignment and test scheduling in relation to the last week of classes, and grades. There is variability in who writes the syllabus across the university, ranging from individual instructors for smaller courses to course leads for large or multi-section courses with teaching teams. It is noteworthy that in some cases the instructor who teaches a section of the course and interacts with students may not be the author of the syllabus.

Developing and Testing the Syllabus Rubric

Two members of the research team (RCT and LBG) developed the rubric by generating a list of potential dimensions based on scholarly literature, drafting descriptions of scale levels for each criterion, and then testing the syllabus rubric on a set of course syllabi from different disciplines.

In the first step of developing the syllabus rubric, we used a deductive approach (Galman, 2013) to compile a list of inclusive and equitable course design and teaching practices based on scholarly literature² that would be broadly applicable across different disciplines and teaching contexts, prioritizing the teaching strategies and practices over the content being taught. We consulted overviews of inclusive teaching practices (Addy et al., 2021a; Center for Research on Learning and Teaching at U-M, 2021; Dewsbury & Brame, 2019; Salehi et al., 2021) as well as scholarly literature on:

• course design and alignment (Fink, 2013; Wiggins & McTighe, 2005),

• transparent teaching (Transparency in Learning & Teaching Project, 2024; Winkelmes et al., 2016),

• structure (Eddy & Hogan, 2014; Tanner, 2013),

• active learning (Ballen et al., 2017; Dewsbury et al., 2022; Theobald et al., 2020),

• belonging (Rainey et al., 2018; Walton & Cohen, 2011),

• Universal Design for Learning (CAST, 2024), and

• motivation (Canning et al., 2018; Harackiewicz et al., 2016; Eccles & Wigfield, 2002).

We then refined this list to select course design and teaching practices relevant to course syllabi, developing a long list of potential criteria for the syllabus rubric.

While we developed most potential criteria based on the scholarly literature, we recognized that some qualities of inclusive and equitable teaching could be more abstract or not found in existing rubrics. This applied to the principle of critical engagement with difference, which involves acknowledging students’ different identities and experiences. To address this limitation, we also took an inductive approach (Galman, 2013) to generate criteria by reviewing three learner-centered test syllabi from different institutions and disciplines (natural science, engineering, social science/humanities). Through this process, we developed four criteria that aligned with this principle: affirming student diversity, diversity among ideas, variety in assessments, and opportunities for autonomy and choice in assessments.

To draft descriptions of all potential criteria, we evaluated existing syllabus rubrics on learner-centered syllabi. Our goal was to identify criteria that illustrate inclusive and equitable teaching practices, such as: alignment between intended learning outcomes and assessments; qualities of formative and summative assessments; course policies and their rationale (Hershock et al., 2022; Palmer et al., 2014); emphasis on classroom community; opportunities for peer collaboration and instructor support; and communication of relevance of coursework (Cullen & Harris, 2009). At this stage, we drafted descriptions for each criterion at each level of the scale, ranging from 0 (not present/assessable) to 3 (present and clear). We then discussed and iterated on the observable characteristics to clearly differentiate between the levels of the scale.

Once we had a working draft of the syllabus rubric, we (RCT and LBG) separately tested it to evaluate three syllabi from different disciplines.³ These test syllabi were drawn from different institutions and disciplines (natural science, engineering, social science/humanities) and used by our CTL in our programming on syllabus construction. We noted any criteria that were difficult to evaluate or that overlapped significantly with other criteria. Following our independent assessments, we compared ratings and discussed each criterion with the aim to adapt the rubric to hone in on concrete, observable characteristics for each level of the scale and reach consensus on criteria we rated differently. At this stage, we continued to refine the working draft of the syllabus rubric. In some cases, we merged multiple criteria to simplify coding while retaining important distinctions between principles of inclusive and equitable teaching. For example, we combined a criterion that focused on belonging (e.g., encouraging and inviting students to engage with instructors) with one that focused on structure (e.g., specific methods to engage with instructors beyond office hours) because both addressed student-instructor engagement. Our goal in this process was to prioritize and maintain measurable characteristics of inclusive and equitable teaching principles that could provide actionable feedback on how and where to revise syllabi.

The Inclusive and Equitable Syllabus Rubric

The syllabus rubric consists of 22 criteria, categorized into four principles based on the organizing framework from the Center for Research on Learning and Teaching at University of Michigan (2021), which also aligns with the way we present and discuss inclusive and equitable teaching in our CTL programming:

• Transparency involves clearly communicating why students are learning course concepts and skills, how learning activities and assessments help them learn, and what students need to do to succeed. It also involves explaining the reasoning underlying course policies. [10 criteria]

• Academic Belonging refers to students’ sense of being accepted and valued in academic classrooms by building relationships with peers and with the instructor(s) and teaching staff and by recognizing the relevance and value of their coursework. [3 criteria]

• Structure (sometimes referred to as Structured Interactions) describes a systematic approach to designing course elements and activities to help students learn and apply concepts and skills as well as protocols or processes that support equitable opportunities for students to participate and interact in the classroom. [5 criteria]

• Critical Engagement of Difference acknowledges and responds to students’ different identities, experiences, perspectives, strengths, and needs by affirming the value of diversity, recognizing different perspectives and ways to solve problems, and allowing for different pathways for successful learning in the course. [4 criteria]

Each criterion was rated along four levels: not present (0), present but unclear (1), present and somewhat clear (2), and present and clear (3). Each criterion was accompanied by a detailed, qualitative description of observable characteristics at each level (see Table 1 for selected examples of a subset of criteria and their descriptive levels). See Appendix for a list and brief description of all criteria.

There is conceptual overlap for some criteria across the four principles. For example, the criterion “opportunities for student engagement with instructor” is categorized under the principle of belonging based on the literature indicating the importance of students feeling connected to instructors to support their sense of belonging (Polmear et al., 2024; Rainey et al., 2018). However, the qualitative description of the criterion also reflects concrete, defined opportunities for students to interact with instructors (e.g., required meetings with instructors during class time or using piazza or slack to ask questions), aligning with the principle of structure.

We also decided to integrate criteria that align with the U-M principle of flexibility into those of structure and critical engagement with difference. We based this decision on conceptual overlap of these criteria with other principles and a desire for parsimony. For example, building in opportunities for student choice in assessments provides students with flexibility in how they demonstrate their learning while also showing responsiveness to students’ different strengths and needs, aligning with the principle of critical engagement with difference. Conceptual overlap and criteria that align with flexibility are noted in the Appendix and in the full syllabus posted on the website. This decision was also based, in part, on institutional and disciplinary considerations. As a STEM-focused institution, we considered that some STEM disciplines emphasize technical accuracy, precision, and minimal tolerance for numerical error (Villanueva et al., 2018). We anticipated that some faculty within STEM disciplines may question the applicability of criteria associated with the label of flexibility. Based on preliminary feedback from faculty colleagues in STEM disciplines, we found that integrating flexible teaching practices into other principles allowed us to more effectively promote inclusive practices that involve adapting to student needs. We note that this decision fits with our institutional context. In other institutions, disciplines and/or educational contexts the framework may be adapted to include flexibility as a stand-alone principle.

Syllabus Analysis

In the first two years of the project, we applied this process to analyze 17 syllabi from focal courses across three undergraduate departments and 12 syllabi from core courses in one graduate degree program. The four departments we partnered with represent different academic disciplines (one science, two engineering, and one social science).

Collecting Syllabi. To identify focal courses in each department or program, we guided a discussion with departmental leadership about the curriculum. We defined focal courses as those that are routinely offered by the department, and may include (1) large enrollment courses and/or introductory courses that drive student interest in the major, (2) core classes that most or all students majoring within the department are required to take, and (3) courses that they feel might be critical to the culture of inclusion and equity in the department (e.g., courses that draw students with highly varied levels of preparation, prior knowledge, backgrounds, or interests). We pulled a list of potential courses from the department website and course schedule and asked for their feedback and additions.

To collect syllabi for coding, a member of the research team contacted departmental administrators, course leads, and individual instructors to request syllabi from focal courses, with permission from instructors. Next, a research team member not involved in future rounds of coding collected class-level attributes from the syllabi. This included department, student level (e.g., lower/upper division), type of class (e.g., lecture, lab), class size and enrollment, credit hours, etc. This team member also deidentified the syllabi by redacting any text identifying instructors and/or teaching assistants and staff to reduce the likelihood of bias affecting our ratings. Once syllabi were deidentified, we used Dedoose software (2024) to conduct two additional rounds of coding: descriptive and evaluative.

Descriptive Coding. The goal of descriptive coding was to capture relevant syllabus excerpts and surrounding context for later evaluation. Two coders were assigned to each criterion. The primary coder took an initial pass through all syllabi, focusing on identifying evidence for one criterion at a time, and flagging excerpts of the syllabi that were relevant to that criterion. Subsequently, the secondary coder went through the syllabi and flagged (1) excerpts that they believed the primary coder had missed and (2) excerpts flagged by the primary coder that they believed were irrelevant to the criterion. The primary coder then reviewed any flagged excerpts to accept suggestions by the second coder or note discrepancies that should be discussed. In general, we encouraged coders to err towards flagging more excerpts to be considered in the evaluative coding stage; accordingly, most suggestions by the second coder were accepted, with discussions only occurring when the two coders appeared to have different interpretations of the rubric that indicated a need to clarify language in criterion descriptions.

Evaluative Coding. The goal of evaluative coding was to review the excerpts identified during descriptive coding and determine an overall score for each syllabus for each of the 24 criteria. Two coders were assigned to each criterion and worked simultaneously but independently, to assign ratings for one criterion at a time across all the syllabi. Each coder reviewed all instances of evidence flagged within one syllabus for a given criterion and assigned a rating of 0, 1, 2, or 3 based on the cumulative evidence:

• A score of 0 indicated that there was no evidence present in the syllabus for a given criterion.

• A score of 1 indicated that evidence of the criterion was present but unclear.

• A score of 2 indicated that evidence of the criterion was present and somewhat clear.

• A score of 3 indicated that evidence of the criterion was present and clear. Overall, scores of 3 were reserved for exemplary examples of each criterion.

In the first year of the project, the inter-rater reliability for evaluative coding prior to resolving discrepancies was high (Cohen’s kappa between 0.80 and 0.89; McHugh, 2012). In the second year of the project, we trained two new coders to assist with coding. The inter-rater reliability was more variable but still in the acceptable range (0.59 to 0.90).

After all syllabi were evaluated by both coders independently for a given criterion, the coders met to compare and discuss their ratings and resolve any discrepancies. If they could not reach an agreement, a third coder made the final decision. Coders noted any clarifications, guidelines, or rules they applied during discussions in the syllabus rubric. The coders also recorded any criteria that were difficult to evaluate or cases where there was minimal variability in the ratings. This process was repeated until all syllabi had scores for all criteria.

Editing of Syllabus Rubric during Coding. During the descriptive and evaluative coding phases, we adjusted the criteria descriptions to improve clarity and, by extension, the consistency of rating. We also refined differences between the four levels of rating for each criterion based on varied qualitative and quantitative differences in observable characteristics (see Table 1), such as:

• The clarity and level of detail in the descriptions in the syllabus,

• The frequency and consistency of the criterion, and/or

• The presence of an additional quality in higher levels.

In one case, we developed a new criterion to measure an element of structure that appeared in syllabi but was not captured in the existing criteria (at the time, 21) of the rubric. This addition occurred during the project’s second year⁴ when we observed that many syllabi described class activities and formative assessments that allowed students to practice with concepts and skills. In response to our observation, we created the criterion “embedded opportunities for practice.” If we modified the rubric during or after coding syllabi, we repeated the descriptive and evaluative coding phases to apply the modified criteria.

Data-sharing Discussions with Departments

A core element of our project was to engage department partners in collaborative meaning-making by holding data-driven discussions about how to advance inclusive and equitable teaching in their departments and teaching contexts. We tailored each departmental discussion by considering both the themes that emerged from the data and the potential for actionable policy and/or pedagogical changes. This adaptive, contextual approach was done after consulting with departmental leadership to understand disciplinary norms, priorities, and the departmental and teaching context. For example, prior to the start of a new semester, we worked with a curriculum specialist in one department to present findings in their faculty meeting and to identify recommendations for syllabus revisions based on our findings. The curriculum specialist then shared exemplars from departmental faculty and templates for syllabus sections that aligned with our recommendations, such as clear policies and pathways for late work submissions, other learner-centered policies (academic integrity, mental health, inclusion), and assessment descriptions that aligned with intended learning outcomes. This approach allowed faculty to see, copy, and adapt exemplars and templates from their colleagues as they revised their syllabi for the upcoming semester.

In another department, we presented major findings and facilitated a discussion with faculty as they constructed and revised shared departmental policies for the next academic year. We drew on data from our syllabus analysis to summarize how class participation was framed in formal communication channels. We also presented data we collected on student perceptions of class participation from a survey distributed to student majors (one of the other data sources collected in the project as a whole). Then, we unpacked data-driven implications for both syllabus revision and teaching practices. We posed questions to foster discussion with faculty about the relevance of participation as a disciplinary skill relevant to students’ future careers. In the context of the discussion, we highlighted the importance of increased transparency with students about the importance of participation, along with structured support to provide multiple ways for students to participate and to help them develop the skills to share their ideas aloud.

Together, these examples demonstrate how we leveraged department-specific data and collaborative discussions with department partners to adapt to distinct departmental contexts and tailor our support to each department, ultimately advancing inclusive and equitable teaching.

Collaboration with Graduate Students

Our collaborative efforts to develop the syllabus rubric and to qualitatively analyze course syllabi provided a fruitful opportunity to train graduate students in educational research methods and build their expertise in the scholarship on inclusive and equitable pedagogy while positioning them as equal partners in educational development. Through a structured fellowship program that involved a commitment of ten hours a week for an academic year, trained graduate fellows played a central role in shaping the rubric and conducting the analysis of syllabi. Inspired by programs that position students as pedagogical partners (Cook-Sather et al., 2021), we worked closely with graduate student fellows during each of the two academic years of the project. One graduate fellow collaborated on the development of the syllabus rubric and both fellows analyzed syllabi and reported findings from the syllabus analysis. The process of developing the rubric and analyzing syllabus data required us to translate principles and concepts of inclusive and equitable pedagogy from scholarly research into specific, measurable characteristics observed in course syllabi. Together, we reckoned with nuances in how we defined, interpreted, and measured each criterion in the rubric as we discussed discrepancies during the coding and analysis of syllabi. These experiences reinforced graduate students’ conceptual understanding of inclusive and equitable teaching principles and practices as well as their skills in synthesizing academic research for application in educational settings.

The collaborative process between members of the CTL and graduate students reinforced the value of integrating and prioritizing student perspectives in advancing inclusive pedagogical practices. Collaborative discussions during the syllabus analysis allowed us to consider perspectives rooted in both the student and instructor experiences. Staff members of the CTL involved in the syllabus analysis were likely to view the syllabi from an instructor’s perspective, based on their extensive experience teaching in higher education and working with teaching faculty. Moreover, as we evaluated syllabi from outside our disciplines, the collaborative coding process with colleagues with different disciplinary training helped us recognize when our own disciplinary contexts and biases informed our perceptions of the syllabi. Collectively, the varied perspectives we brought to the syllabus analysis allowed us to recognize and question when we used our role in the classroom or disciplinary perspectives to inform what we prioritized in our discussions and reports for departmental partners. By collaborating with graduate students and with colleagues from different disciplines, we were able to consider and integrate multiple perspectives during the coding and the reporting of strengths and recommendations on the syllabus analysis.

Applications and Recommendations

Course syllabi are an easily accessed source of rich data on course design and the instructor’s intent to implement inclusive and equitable teaching practices. Given the utility and accessibility of course syllabi, the rubric we developed can be applied at various levels to identify individual, departmental, and school- or university-wide patterns in course design and inclusive pedagogy. In our project, we developed the syllabus rubric as a tool to identify indicators of inclusive and equitable teaching across focal courses in an academic department. We synthesized data from the syllabus analysis with results from a student survey and instructor interviews. These data from multiple sources informed our discussions with departments about how to advance inclusive and equitable teaching across the curriculum and teaching contexts.

In our application of the syllabus rubric, we gathered and synthesized data at the department level to focus on a “key unit of change” (Reinholz & Apkarian, 2018, p. 1) at our decentralized institution. Scholarly literature on change theories point to academic departments based on the relative coherence of departmental cultures (relative emphasis on teaching vs. research, value of teaching innovation) and structures (e.g., teaching loads, process for making teaching assignments, promotion and tenure processes; Reinholz et al., 2019). However, the scope of the analysis and the level of data reporting may vary across institutional contexts and departmental structures. At more centralized institutions, aggregating data across schools or even campuses may be more relevant and compelling levels of analysis. To evaluate the appropriate scope for data collection and analysis, researchers and educational developers may consider the coherence of teaching-related structures, educational mission and values, and teaching cultures within and across departments and other academic units.

Going beyond the applications in the current project, we envision the inclusive and equitable syllabus rubric as a useful tool for:

• instructors to reflect and iterate on the elements of inclusive and equitable teaching in their courses,⁵

• educational developers to engage in data-informed consultations,

• department leadership to identify patterns of inclusive and equitable teaching across the curriculum,

• disciplinary organizations to highlight patterns and trends in course design and inclusive pedagogy, and

• CTLs to identify needs for programming and support with instructors and departments.

We encourage educational developers to borrow or adapt the syllabus rubric to their institutional contexts. Importantly, the syllabus rubric was designed as a tool for formative feedback to inform next steps for teaching development and for change efforts at the departmental level, in contrast to a tool that provides a summative evaluation of the quality of teaching and/or course design. Accordingly, the applications we recommend emphasize the syllabus rubric as a source of ongoing formative feedback to guide teaching development.

Limitations of the Syllabus Analysis

There are two primary limitations associated with using the rubric to analyze syllabi. The first has to do with the inherent limitations of the syllabus itself: it is an indirect communication from the instructor (or author of the syllabus) rather than a direct reflection of the actual teaching happening in the course. For example, descriptions of class sessions may reflect an instructor’s intent to implement active learning activities, while the actual class sessions may shift to focus on content coverage through lecture. Moreover, there may be variation in who writes the syllabus and how much autonomy they have over the learning outcomes, assessments, and policies. Some institutions may require instructors to include standardized policies (at the university or department level). For example, an instructor may include a strict department-wide policy on attendance on the syllabus but choose not to enforce that policy in practice. In such cases, these elements of the syllabus may not reflect the instructor’s intent and/or how they carry out policies or inclusive and equitable teaching practices in the classroom. Given these limitations, data gathered from the syllabus can prompt further questions and discussion to understand the context and unpack implications that advance the use of inclusive and equitable teaching practices.

Additionally, a syllabus is not always the primary method of communication of course-related information. This information may be presented verbally in class or stored on the course learning management system (LMS). These limitations are part of why the project integrates data from both students and instructors prior to drawing strong conclusions. During discussions with departments, we acknowledged the variety of communication channels beyond the syllabus and used it as an opportunity to be curious rather than make assumptions when criteria were absent or lacking detail in the syllabi.

A second limitation is that the syllabus rubric itself may not capture all indicators of inclusive and equitable teaching, especially considering the multifaceted definitions of inclusive and equitable teaching in the scholarly literature. We expect the syllabus rubric will continue to develop as its use expands to different types of courses and programs and the changing needs in various institutional contexts. We experienced this evolution of the rubric during both the initial development and the second year of using the rubric in a new academic program. Specifically, we noted many structured opportunities for practice in the syllabi that were not being captured by the rubric but that were important, evidence-based components of inclusive and equitable teaching. After discussion within the team, we decided that a new criterion needed to be added to the rubric: embedded opportunities for practice. Drawing on Addy et al.’s (2021a) description of the syllabus as a “living constitution,” this syllabus rubric “invites participation, allows for evolution (or amendment), and accommodates a community’s changing needs,” (p. 50). We encourage adopters of our syllabus analysis tool to adapt the rubric to work in their context and contact us with ideas and modifications to support ongoing iteration and refinement of the rubric.

One potential area for expansion of the syllabus rubric is to engage more explicitly with practices based in critical pedagogy (Freire, 2000; hooks, 1994) and other pedagogical approaches with an anti-oppressive lens.⁶ Consistent with these educational philosophies, the existing rubric includes criteria that emphasize the participatory (e.g., embedded practice opportunities, autonomy in assessments) and relational elements (e.g., student-instructor engagement, peer support and collaboration, participation structure and variety) in the syllabus. However, additional criteria could be added to emphasize opportunities for students to provide input or make choices about learning activities, assessments, and/or course materials (i.e., co-creation in teaching and learning; Bovill, 2020). Moreover, future iterations of the rubric could incorporate undergraduate student perspectives to inform revisions to existing criteria, create additional criteria, or provide instructors and departments with targeted feedback on syllabi. The current rubric incorporated input from a graduate student as an equal partner in its development; however, feedback from undergraduate student focus groups or pedagogical partners (Cook-Sather et al., 2021) would be a valuable next step in the iteration and refinement of the syllabus analysis tool.

Additional Considerations

One essential consideration in the analysis and reporting of data from course syllabi is the variability of contexts across individual instructors, departments, disciplines, and institutions. Accordingly, our team prioritized conversations with members of the department to understand those contexts and use them to inform our approach to analysis and reporting. For example, we grappled with how best to share data with departments as we felt a tension between more comprehensive reports versus more accessible, brief summaries of the data we collected. As researchers, we wanted to be thorough and detailed in how we reported data. As educational developers, we wanted to ease reading and processing of the reported data to help faculty with limited time recognize the main points and implications of the data for advancing inclusive and equitable pedagogy.

In the pilot year of the project, we provided comprehensive written reports to departments that included scores on each syllabus criterion as well as strengths and recommendations based on patterns observed across departmental syllabi. This approach was time consuming and resulted in reports more than ten pages in length for just the syllabus analysis. In the next round of the project, we consulted with department leadership about the appropriate audiences, level of detail, and purpose of the reports. For example, in one department we led leadership through a comprehensive presentation of the major findings and later presented to faculty about specific teaching recommendations informed by the data. A primary strength of the project was its tailored approach to departmental norms, which also informed how we presented and discussed the findings from the syllabus analysis.

Syllabi are a primary data source in our comprehensive project aimed at facilitating departmental change efforts, enabling access to rich information about inclusive and equitable teaching that can be readily compared across disciplines and teaching contexts. We hope that other educational developers, researchers, and readers across roles will adapt the syllabus rubric as a tool for formative assessment to advance inclusive and equitable teaching in their contexts.