Research Site and Participants

Researchers gathered syllabi from 110 faculty across the two residential campuses of Embry-Riddle Aeronautical University (ERAU), a private, STEM-focused, predominantly white institution located in the Southwestern and Southeastern United States. The Daytona Beach, Florida, and Prescott, Arizona, campuses combined enroll approximately 10,000 students and employ approximately 700 faculty. The 110 faculty participants were randomly selected from 190 eligible faculty who had taught the same course pre-, mid-, and post-pandemic (Fall 2019, Fall 2020, and Fall 2022, respectively).

Data Collection

A statistical power analysis was performed to identify a reliable sample size. The effect size in this study was 0.5, recommended by G*Power software. With an alpha = .05, power = 0.80, and allocation ratio = 1, the projected sample size needed was approximately N = 102 overall for this independent samples t test. The proposed sample size allowed for expected attrition and to control for possible mediating/moderating factors, subgroup analyses, and other analyses. Researchers selected 110 faculty to ensure an appropriate sample size.

Syllabus data for the 330 syllabi (for 110 faculty, across all three time periods) was de-identified and assigned a numeric code. Four faculty (two from each campus) were hired as independent scorers, along with the researchers. Scorers participated in two 1-hour calibration sessions, using sample syllabi not in the data set to norm scoring using the five-component rubric.

Syllabi were not separated by time period for scoring; instead, syllabi were randomly assigned to two raters. If the two scores for a rubric component conflicted in the extreme (scores of 0 and 2), that component’s scores were replaced by a third rater’s score. Replacing the score rather than taking the mean in cases of extreme discrepancies helped to prevent an inflation of Type 1 error and ensured that results were not misleading due to compromised averages not accurately reflecting any rater’s perspective.

Data Analysis

A PCA and a one-way within-subjects MANOVA were conducted. For the MANOVA, variables consisted of five dependent variables (i.e., Likert-scale question ratings for each syllabus) and three independent variables (i.e., Times 1, 2, and 3). Descriptive results portrayed in Table 1 include the means and standard deviations for each question and rating time. Means range from a high of M = .9682, SD = .63635 for the Question 1, Time 2 rating and a low of M = .4227, SD = .53410 for the Question 4, Time 3 rating.

Principle Components Analysis and Reliability Analysis

The PCA was run with an orthogonal varimax rotation. Initial results reveal a Kaiser-Meyer-Olkin measure of sampling adequacy (KMO) value of .873, which is considered very good, as it indicates the proportion of variance in variables that might be caused by underlying factors. Furthermore, within the PCA results, Bartlett’s Test for Sphericity shows that sphericity has been violated and all variances are not equal to one another, χ2(105) = 1109.09, p = .000, suggesting that the variables in the data set are intercorrelated and may not be ideally suited for a straightforward PCA.

To address this, other considerations may be to employ alternative dimension reduction techniques, such as using oblique rotation methods within PCA or investigating the variables to understand and potentially remedy the high intercorrelations. Screen plot results are interpreted to have five major components, according to the five Likert-scale rating questions. Upon further evaluation of the rotated components matrix (Appendix B, Table 2), it was found that the variables that are supposed to represent a specific question did not load highly on those components (i.e., Likert-scale rating question).

More specifically, for Component 1 (Question 1), Variable Q2_T3 (Question 2, Time point 3 ratings data) loads highest; for Component 2, Variable Q4_T2 has the highest loading value; for Component 3, Variable Q5_T1 loads highest; for Component 4, Variable Q1_T2 has the highest loading value; and for Component 5, Variable Q2_T1 loads highest. In other words, the variables (rating results for each of the questions) do not appear to match with the question they are supposed to be measuring.

Overall, the results from the PCA offer a nuanced view of changes in the student-centeredness of syllabi. The mismatch between expected and actual patterns, where variables didn’t align neatly with the questions we thought they would, reveals that changes in syllabi may not be straightforward. In practical terms, this analysis helps us understand that the pandemic has led to many faculty reevaluating and adjusting syllabi, with some changes likely to persist due to their effectiveness or necessity. At the same time, the resilience of certain pre-pandemic syllabus practices suggests that not everything has changed. The ongoing influence of the pandemic might mean that faculty continue to adjust, reflecting a new normal that is not static, but instead characterized by ongoing adaptation and reevaluation of practices.

Multivariate Data

Sphericity is the equality of variance of the differences between each pair of values. Mauchly’s Test of Sphericity was interpreted for the current analysis, showing sphericity has not been violated for all question measures, except for Question 5, which was found significant and therefore sphericity has been violated, p < .05, with a Greenhouse-Geisser value of .888. This value being less than 1 means that the test of sphericity was not met, and therefore the F value for further analysis results may be inflated.

Wilks’ lambda is a value that ranges from 0 to 1 with a value closer to 1 indicating stronger evidence that the explanatory variable has a statistically significant effect on the values of the response variables. For the present results, found in Table 3, Wilks’ lambda showed an insignificant effect between time ratings Λ = .849, F (10,100) = 1.783, p = .073.

Based on the multivariate results, significant values found in the test of within-subjects contrasts table (Table 4) and the pairwise comparisons table (see Appendix B) may be inflated.

The test of within-subjects contrasts table shows significant results for time ratings within Questions 1, 3, and 4. Overall, partial eta squared (ηp2) values are very low, showing little variance being accounted for from the analysis.

Within the pairwise comparisons table, significant results are indicated when p < .05. Significant results indicate that those variable results are different from one another. Therefore, within Appendix B, pairwise comparisons show significance between:

• Question 1 ratings for Times 1 and 2, and between 2 and 3;

• Question 2 ratings for Times 2 and 3;

• Question 3 ratings for Times 1 and 2, and between 2 and 3; and

• Question 4 ratings between Times 2 and 3.

Differences in ratings for Questions 1 through 4 show significant differences in rating from Times 1 and 2, and 2 and 3, representing an upside down “U” curve, which can be seen for Question 3 in Figure 4.

Figure 4. Question 3 Time Ratings Curve

Between-Subjects Effects

Findings comparing the overall results from each of the questions show significant results. Results found in Table 5 portray significant F and partial eta squared values, meaning that each question’s overall ratings significantly differ from one another, and the interpreted results account for over 50% of variance.

The MANOVA results offer insightful clues into how syllabi have shifted during the pandemic and whether these changes have staying power. First, the issue with sphericity, where variances among some measures weren’t equal, suggests that the data for Question 5 (high expectations and confidence in students through hard work) behaved differently from the others. This discrepancy indicates that the impact of the pandemic on various aspects of student-centeredness didn’t follow a uniform pattern; some changes were more pronounced or variable than others. Additionally, Wilks’ lambda told us that, broadly speaking, the differences in syllabi over time weren’t statistically significant across the board. This suggests that changes may not be as sweeping or uniform as one might expect. However, significant results in specific pairwise comparisons, particularly the changes in ratings from one time period to another for certain questions, highlight that there were indeed notable shifts in some areas of syllabi. These significant pairwise differences, especially where we see an “upside down U” curve for some questions, indicate that some representations of student-centeredness initially changed quite a bit as the pandemic began, then perhaps settled into a new pattern or returned closer to pre-pandemic methods over time. This pattern could reflect an initial rush to adapt to remote teaching or other pandemic-related changes, followed by a period of adjustment and settling into more effective or sustainable practices. The significant differences found in overall ratings for each question, and the fact that these differences account for a substantial amount of variance, reinforce the idea that the pandemic has had a tangible impact on syllabi. However, the specific nature of these impacts seems to vary by question, suggesting that some areas were more affected than others.

To delineate faculty learning changes into distinct cohorts, specific threshold values were employed. After calculating the differences between Time 1 and Time 2 (ΔTime1-Time2), Time 2 and Time 3 (ΔTime2-Time3), and Time 1 and Time 3 (ΔTime1-Time3), four cohorts were established:

• Increase – Those faculty members whose scores consistently demonstrated improvement over time, indicated by a ΔTime1-Time2, ΔTime2-Time3, and ΔTime1-Time3 greater than 1 (N = 28).

• Decrease – Faculty participants displaying a consistent decline in scores, reflected by a ΔTime1-Time2, ΔTime2-Time3, and ΔTime1-Time3 less than –1 (N = 28).

• Static – Participants whose scores exhibited minimal variation over time, falling within the range of -0.5 to +0.5 for ΔTime1-Time2, ΔTime2-Time3, and ΔTime1-Time3. This cohort was further subdivided into two subcohorts: “Student-Centered” and “Content-Centered.” The Student-Centered cohort encompassed individuals whose scores exhibited sustained high scores despite negligible changes over time (n = 4). The Static, Content-Centered subcohort pertained to individuals whose scores remained consistently low (n = 11).

• Elastic – Individuals whose scores increased then decreased, or vice versa. The Increase to Decrease subcohort were classified by a ΔTime1-Time2 equal to or greater than +1 and a ΔTime2-Time3 equal to or less than -1 (n = 27). The Decrease to Increase subcohort were classified by a ΔTime1-Time2 equal to or less than -1 and a ΔTime2-Time3 equal to or greater than +1 (n = 12).

An overview of all faculty cohorts can be found in Table 6. Note that positive changes in student-centeredness describe 50% of syllabi measured; however, these changes did not persist for 25% of faculty syllabi measured. For the Elastic cohort, representing over one-third of the faculty, results suggest that the pandemic was a significant disruptor that had a temporary impact on syllabi.

Notably, a MANOVA, conducted to assess the interaction between time and several independent variables (rank, campus, college, the type of student-centered syllabus provided, and participation in a short training course), revealed that among these factors, the interaction between time and participation in a short training course on student-centered syllabi was the only one to show statistical significance. This interaction effect was quantified using Pillai’s trace, which yielded a value of .342. The corresponding F value for this interaction was 1.7, indicating the ratio of variance explained by the model in comparison to the variance unexplained. The significance value (p value) associated with this interaction was .031, falling below the threshold of .05, which suggests that the interaction effect is statistically significant. This implies that changes over time in the dependent variables are significantly influenced by the participation in a short training course.

Why might an instructor be elastic in their practices?

Early analysis suggests faculty could be elastic in their practices for many reasons. The researchers are conducting interviews with select faculty to identify potential driving and resisting factors to changes in practice. Pandemic practices created a heavier workload for most, which faculty could be eager to unload once campuses returned to a more normal set of operations. In addition, faculty less adept with technologies may be choosing to revert to practices that are more efficient or comfortable to them. Local and disciplinary requirements, such as institutional and/or accreditation requirements, also drove both changes in practice and hindered faculty from making changes to practice.

In addition, the political landscape of higher education cannot be overlooked as a possible influence. Faculty and administrators are still navigating the complexities of how local and national politicians are attempting to influence teaching practices during the pandemic (Wippman & Altschuler, 2022). While faculty and students may continue to feel the ongoing impact of the pandemic, institutions could have political mandates and/or incentives to return to pre-pandemic practices (Johnson et al., 2021). Our continued analysis of faculty driving and restricting factors is discovering that the elasticity of instructors’ syllabus practices and policies can be shaped by a multitude of factors, including workload considerations, technological proficiency, and the political dynamics within higher education, which highlights the complex interplay between individual choices and institutional imperatives.

What additional data can inform syllabus analysis?

While analysis of changes in syllabi can offer insights into changes faculty may have made both during and after the pandemic, additional data can help triangulate findings. Faculty interviews, as we mentioned, are helping the researchers identify the values, beliefs, and strategies influencing faculty decision-making during this time. Insights from ongoing faculty interviews promise to shed light on the driving and resisting factors underpinning pedagogical change, essential for fostering sustainable and impactful pedagogical change. In addition, analysis of course content found on LMSs could enrich discussions of student-centeredness of faculty. Perhaps most importantly, student voices can provide an important lens into the extent to which a faculty member’s student-centeredness is felt by students in their courses. Midterm feedback of student opinions is a popular faculty development service that teaching and learning centers can provide. Gathering student opinions over time can provide an important longitudinal complement to syllabus data.

When defining education development, most academics focus on classroom impact through innovative pedagogy and improving teaching; equally important is how to support institutional change (Sorcinelli et al., 2005). Systematic or transformative change in higher education is extremely difficult to enact and sustain (Fink & Stoll, 1998), unless there are driving factors facilitating the change. In this case, the pandemic drove systemic institutional change, particularly related to the learning experience, however our examination of how syllabi evolve over time amid pandemic challenges underscores the diverse responses among faculty. While some continued to embrace student-centered approaches post-pandemic, others reverted to familiar practices, highlighting the elasticity inherent in pedagogical adaptation. Ongoing interviews with faculty members aim to unravel the complex factors influencing these practices. As we delve deeper into these driving and resisting forces, it becomes evident that understanding pedagogical change requires a multifaceted approach, one that integrates a variety of sources. By triangulating these data, we can identify sustainable pedagogical transformations, ensuring the resilience and efficacy of our educational practices in the face of uncertainty.

ERAU’s Institutional Review determined that no IRB review was required for this research on 12/8/23.