CDI Impact

Gravett and Broscheid (2018) detailed models and genres of educational development, noting the versatile expanse of programming available. Drawing from the work of Hurney et al. (2016), Gravett and Broscheid contended that the specific type, format, or approach utilized (e.g., consultation, workshop) must be nested in faculty learning outcomes that also support student learning outcomes and, plausibly, other institutional or community goals. Should an instructor wish to develop a new service-learning course for undergraduates that necessitates a long-term community partnership, for instance, a short, one-time workshop may be ineffective (Gravett & Broscheid, 2018). Instead, “if the desired outcomes of faculty development … revolve around lasting changes, then more extended, immersive programs may be needed” (p. 100). Institutes provide such depth, typically committing participants to a multiday structure in which they work both individually and with teams on a common aim (e.g., course design).

With support from educational developers, peer mentors, and sometimes student consultants, CDIs specifically “aim to help instructors create rich, active, supportive classroom environments grounded in evidence-based practices; expand their pedagogical content knowledge; become reflective practitioners; and foster teaching community and personal growth” (Palmer et al., 2016, p. 1). Research on the extent to which CDIs promulgate development along these lines, however, is underrepresented in the literature. In a meta-analysis of 138 studies on the impact of educational development, Chism et al. (2012) reviewed 49 studies on institute efficacy; these are largely limited to results focusing on participant satisfaction and perception—for institutes of one day or more, findings pointed to “positive effects on teaching attitudes and changes in teaching practices” (p. 135). Relatedly, they located 20 studies on formal courses in teaching. With some exception, this body of research documents the impact on participants’ thinking about teaching but lacks insight into other impact factors.

More recently, Palmer et al. (2016) published the first systematic study of CDIs—one that richly informs our awareness of short- and long-term impact. Employing a multifaceted, mixed methods approach, they analyzed post-institute satisfaction and perception data, pre- and post-institute surveys on pedagogical confidence, pre- and post-institute self-report tools probing ability to craft learning-focused courses, and pre- and post-institute syllabi. Situating their work in Kreber and Brook’s (2001) impact model, the researchers sought to ascertain multiple points of impact beyond that of participant satisfaction alone. Salient takeaways included the following: participants learned the basic principles of course design, pedagogical confidence (i.e., participant beliefs about teaching and learning) was bolstered, and instructors’ syllabi (post-CDI) favored learning-focused components over content-centric elements (Palmer et al., 2016).

Research into participant perception and satisfaction, beliefs and attitudes, and post-institute applications adds significant value to our understanding of the impact of CDIs on educational development. Absent from the nascent literature on CDIs is a focus on the intersecting benefits and costs of institute engagement on faculty outcomes. Incisively noted by Gravett and Broscheid (2018), “Crafting high-level, applied, authentic learning experiences may be a daunting task for faculty” (p. 87). Designing and implementing a new course does not transpire in a vacuum; consider the array of other activities vying for a faculty member’s focus (e.g., research development and grant procurement, promotion and tenure, service and outreach expectations). To this end, one wonders if the positive impacts of CDIs are indeed sustainable and what, if any, limitations accrue for participants.

Motivation Theory

Motivation theory provides a lens through which we can begin to make meaning of this quandary. Although theories of motivation abound, the work that has been done in educational development does not offer a theoretical framework for understanding faculty motivation to participate in a CDI (Lowenthal et al., 2013). Of the range of available options, expectancy-value (EV) models of motivation stand out “for their ability to synthesize multiple theoretical perspectives, capture the key components of what motivates an individual, and explain a wide range of achievement-related behaviors” (Barron & Hulleman, 2015, p. 1). Furthermore, Eccles et al.’s (1983, as cited in Flake et al., 2015) EV model of motivation has been widely used to understand learner experiences (e.g., academic choices, performance behaviors, student attitudes) in an educational context. In the case of CDIs, faculty participants are the learners of interest, rendering EV a suitable lens through which we may understand facets of the faculty experience of CDIs.

A recent amendment to EV theory is Hulleman and Barron’s (2015) expectancy-value-cost (EVC) theory of motivation. EVC theory focuses on three factors central to a comprehensive understanding of overall motivation: having an expectation that one can be successful in a task (i.e., expectancy), placing value in that task (i.e., value), and recognizing the costs associated with that task (i.e., cost). Although the original theory included cost as a sub-factor, Hulleman and Barron theorize that cost is a third central factor necessary to understanding motivation. To date, however, cost remains under-theorized, and little empirical measurement has been done in this area (Flake et al., 2015).

Overview of the Model

There are four important features of the EVC model of motivation. First, the model is psychological, based in participant perceptions and interpretations. It is less important how much actual skill a participant has for performing a particular task and much more important how that individual answers the question, “Can I do the task?” In other words, a participant’s beliefs are more relevant to understanding motivation than actual ability. Secondly, the model is developmental; it asserts that the factors intersecting to create motivation can be shaped over time by personal characteristics, experiences, and context. Third, the model suggests that motivation is additive; per Hulleman and Barron (2015), the formula is represented as follows: Motivation = E + V - C. Lastly, the model synthesizes multiple perspectives to explain comprehensively a range of participant achievements and attitudes.

Expectancy (E) suggests that a learner who believes they can do something is more likely to be motivated to participate in a related behavior or task. Thus, a faculty member who expects to be successful in redesigning a course may be motivated to partake in an intensive institute. The second concept, value (V), means that those who value a task are more motivated to do what it takes to achieve that task. Faculty who do not value pedagogical innovation, or see no reason to alter status quo, are unlikely to register for a CDI. However, participants who receive compensation for engaging in a CDI may be motivated, chiefly or peripherally, by a financial incentive.

The last factor is cost (C): the higher the costs associated with a task are, the lower the participant’s motivation will be. Four types of cost are theorized (Hulleman & Barron, 2015). The first, task difficulty, is the amount of effort required. Take, for example, efforts associated with designing transparent, performance-based assignments that engage students. Effort-unrelated costs reduce motivation due to the amount of time, energy, or resources that a faculty member has to expend on other tasks, such as publishing research or caring for children. The third, opportunity cost, is the loss of engaging in valued alternatives due to participating in the targeted task (e.g., missing an exercise class due to time spent on course prep). Lastly, there are negative psychological consequences, such as fatigue experienced after completing a CDI and realizing the time-consuming need to redesign other courses.

Assessing each of the individual factors within the model is necessary to understand which impacts motivation. For example, a faculty member who is experiencing high cost/low motivation from competing demands to complete a tenure dossier may need different support than one with a fixed, deterministic mindset about teaching. While the three factors are intended to be used under the comprehensive framework of gauging motivation, it is important to differentiate between the contributions that each make in shaping participant motivation.

Intervention

Each summer our CTL offers an intensive CDI. As is common with most of our CTL programs, participants generally find out about the CDI through our weekly all-faculty email digest, which provides information on upcoming initiatives and faculty resources. Through a regional network, we also invite instructors from nearby and regional institutions (e.g., those without CTLs) to participate free of charge. Participants apply to the CDI, stating their rationale for attending the institute and documenting their hoped-for outcomes. Nearly all applicants are accepted into the institute; those who fail to complete the application (or who are unable to commit to the full week) may be denied a spot. Faculty do not receive stipends to attend.

The curriculum focuses on investigating the situational factors influencing student learning; attending to student motivation, engagement, and development; developing course goals and student learning outcomes; creating assessments that align with outcomes; and creating engaged pedagogies and learning activities. The institute meets for five full (i.e., 7- to 8-hour) sequential days. Approximately 20 participants are divided into groups of four to six, each guided by a small-group faculty facilitator. Using an array of evidence-based pedagogies, key elements are taught by lead facilitators. Various readings and activities (e.g., Fink, 2013; Hansen, 2011), given as homework, enrich participants’ progress and model learner-centered design.

Between sessions, participants complete a daily deliverable; these assessments culminate in a course portfolio that is presented in a gallery format during the final day of the institute. For most participants, the portfolio represents a full unit, module, or week of institutional content (i.e., outcomes, in-class and out-of-class assessments, and activities). In rare cases, participants are able to showcase pedagogy, activities, and assessments for an entire course. Notably, our CTL has developed and published a manualized curriculum for the institute, which allows for consistency across institutes and provides participants, post hoc, with review materials. All facets of the institute curriculum, ranging from what transpires daily to the associated rationale and research bases, are made transparent to institute participants. This allows participants to continue to work on their courses post-institute and to later apply what they have learned to future course development opportunities.

Instrument

To understand the full impact of our CDI, we developed a web-based instrument that included a battery of measures composed of existing or adapted published instruments with sound psychometric properties (see Table 1). We chose this method in lieu of developing unique measures, which entails a lengthy process of content and construct validation and pilot testing. Once our battery was compiled, we invited a cadre of faculty developers, general faculty, and doctoral-level assessment and measurement experts to pilot the instrument. This allowed us to ascertain the amount of time one might expect to spend taking the survey (i.e., 15–25 minutes) as well as to ensure questions were both clear and answerable.

For the purpose of this specific article, note that analyses and interpretations were centered on findings related to faculty motivation as measured by the Faculty Attitudes Toward Teaching Scale (Hulleman & Barron, 2012; Hulleman & Schiefele, 2010), which was adapted from the Teacher Interest Survey (see Schiefele, 2009; Schiefele et al., 1993, 2013). The 30-item adaptation is made up of six subscales: expectancy (e.g., I am confident that I can get students to learn), value (e.g., Teaching is a worthwhile career), costs (e.g., Teaching requires me to make sacrifices in other areas of my life), pedagogical interest (e.g., I like learning about new teaching methods), student interest (e.g., Working with students is one of the most enjoyable aspects of teaching), and subject-matter interest (e.g., The topics I get to teach are personally important to me). A complete list of items can be found in the appendix. Responses were recorded on an 8-point Likert scale (ranging from completely disagree to completely agree) and converted to numeric scores (Table 2). The scores for the costs subscale had the opposite sign, as the statements were all negative.

Participants were invited to complete the instrument right before the program (pretest) and during implementation of their (re)designed courses (posttest). The pretest provided baseline data, whereas the posttest captured participants’ longitudinal development. Specifically, the posttest was deployed three-quarters of the way into the (re)designed course, allowing each instructor to determine accurately the ease or difficulty of implementing their course. We also felt we would ascertain a more truthful reflection at this time, as opposed to at the end of the course, which is especially taxing for faculty members. Furthermore, we intended to compare the pretest and posttest in our analysis toward this article’s aims, in order to measure change within the context of a full intervention.

Participants

Across five separate iterations of our CDI, 112 instructional faculty took part. Of these individuals, all but 13 worked for our university, which has an enrollment of approximately 20,800 undergraduates and 1,900 graduate students. At the time of study, the average class size was 25, with 88% of classes having fewer than 50 students. Only 2% of the courses at our university are taught by graduate students or staff. Those CDI participants not employed by our institution arrived from regional, teaching-focused colleges and universities.

Other than college/disciplinary affiliation, we did not collect data on participant demographics (e.g., gender, race/ethnicity, rank). The majority of the participants (28%) were from the humanities and social sciences (which is, already, proportionately large at our university), followed by the behavioral and health studies disciplines (16%), computer science and engineering (14%), mathematics and sciences (14%), and business (11%). Less represented were faculty members from education (6%) and the visual and performing arts (1%). Participants affiliated with other branches of the university (e.g., Student Affairs, Professional and Continuing Education, etc.) represented 9% of our participant pool.

Ninety-five individuals (84.8%) took the pretest, and 49 (43.8%) took the posttest. To ensure participant anonymity, we utilized unique identifiers to match cases across the time series. However, only 32 (28.6%) completed both surveys using the same unique identifier. Among the points of rationale for such a strong initial response rate is that individuals were able to complete the pretest either online (i.e., pre-institute) or at a computer provided during check-in for the institute’s first day. The posttests were made available to participants through an online link distributed through email. Though the interventions were distinct with respect to the year that they were offered and included some variability across annual facilitation teams, the manualized curriculum (i.e., CDI learning outcomes, content, pedagogies and activities, assessments) remained largely intact. An implementation fidelity analysis (Fisher et al., 2014; Smith et al., 2019) was conducted by a doctoral research assistant with affirmative results. This technique ensures that what actually transpires within an instructional intervention is coherent with what was planned, adding validity to our ultimate findings.

What Educational Developers Can Do

Among the many issues we have contemplated in studying faculty motivation are the temporal layers within which the phenomenon is manifest. There are three ways that a CDI (and, perhaps, similar teaching interventions such as learning communities) creates effort for faculty members. First, faculty motivation affects learning how to design a new course; expectancy, value, and cost considerations are relevant before and throughout the CDI experience. Faculty members have to learn the principles of backward design, for instance, and build the skills necessary to apply and integrate pedagogical ideals. Although this was not the province of our particular study, educational developers might pay attention to such considerations when designing CDI curricula, which would include the pre-institute experience (e.g., use of pre-institute surveys, considering methods of conveying expectations, etc.).

A second tier wherein motivation challenges may accrue is in course preparation for which faculty, post-CDI, continue to apply principles learned during the CDI to tasks such as syllabus construction, assessment design, and pedagogical choice-making. Herein, the faculty member is a practitioner—sometimes solitarily—and no longer a learner within the supported, guided CDI context. A final layer closely tied to the aforementioned involves effort related to course delivery itself (i.e., the semester or quarter during which faculty members facilitate their courses amid an array of situational factors). Some factors may be anticipated, such as student demographics, and others less so, such as the unplanned onboarding of courses to online delivery in the wake of the COVID-19 pandemic.

That the faculty in our study expect to teach well, and continue to self-report prowess in this area over time, is not a finding to take for granted. Nor may this finding be transferable to all centers, particularly at institutions classified or structured differently from ours, a master’s comprehensive university with a strong teaching focus. To this end, we have forecasted ways to transpose Barron’s (2018) strategies of bolstering student expectancy to the faculty experience. Expectancy is tied to ability and skill, which points to providing faculty with opportunities to practice and receive specific (versus general) feedback as they develop new courses. Herein, facilitator (i.e., educational developer, peer) encouragement is integral. As learners experience growth, development, and improvement, expectancy increases (Barron, 2018). An appropriate balance of challenge and support is also vital; difficulty should be tied to the learner’s skill level in the context of appropriate support (Barron, 2018). Pedagogically, such tenets, if not already present, can be woven thoughtfully into CDI curricula.

The finding that value toward teaching increases from the pretest to the posttest is one that we recommend exploring qualitatively. Unlike some centers that offer post-institute learning communities to help faculty continue their work after the CDI (i.e., the kinds of interventions that strengthen value), ours does not provide this kind of structured support. It would seem that faculty benefit, just as students do, from the application of and engagement with the pedagogical practices that they have learned through the CDI. Varied positive benefits to faculty of innovative teaching practices are documented in the literature (e.g., Meixner, 2010, 2013), both refereed and gray (e.g., listservs, blogs), and come from multiple disciplinary perspectives and methodologies. We imagine that offering faculty ongoing opportunities to reflect on their post-CDI teaching experiences with a community of peers—face-to-face or virtual, synchronous or asynchronous, informal or formal—will serve to enhance value.

Nevertheless, the initiatives (e.g., post-CDI workshops and learning communities) that boost faculty expectancy and value may come at a cost to faculty, which is a factor of concern that has emerged from this study. We recognize, in reflecting on the various pedagogical strategies introduced during our CDI, that many evidence-based practices—though strongly supported by the literature—are extremely time intensive for faculty to enact (particularly when an instructor is deploying a newly designed course for the first time). We recommend, for ourselves and others, thinking more strategically about teaching faculty about efficiencies in course design—presenting “jewels” or “hacks” that facilitate student learning and the assessment of learning in novel ways. Transparent assignment design (see Palmer et al., 2018; Winkelmes et al., 2019) is among these gems, as are opportunities for faculty to embrace individual, peer, and community assessment (Fink, 2013). If course redesign doesn’t attend to grading efficiency, the result may be idealized courses that cannot be implemented realistically. The overall impact could be demoralizing to faculty and contrary to the goals of course design.

The literature on threshold concepts applied to students, faculty, and educational developers (Bunnel & Bernstein, 2012; Kiely & Sexsmith, 2018; King & Felten, 2012; Meyer & Land, 2006) may provide an additional conceptual pathway to understanding and exploring faculty expectancy, value, and cost. Kiely and Sexsmith (2018) deftly wrote, “Learning of new threshold concepts is an important area for faculty development; because these moments are rarer for faculty, they imply greater dissonance and resultant metacognitive shifts” (p. 288). Like our students, faculty traverse the unknown as they learn, grow, and develop as teachers; at times, they encounter states of bewilderment, confusion, and difficulty—“stuck places”—that compromise their motivation to persevere. Educational developers are advised to use an array of tools to help faculty address and navigate such dissonance, reflecting critically on their assumptions and paving new pathways forward (Kiely & Sexsmith, 2018).

As a point of departure from teaching, but a crucial consideration, is the reminder that emanates from the cost data: faculty are whole people, navigating costs to teaching on a daily, sometimes hourly, basis. Such costs—like the need to finalize a manuscript for publication, the responsibility to participate in committees, the challenges of applying for promotion amid a hefty teaching load, and the disproportionate service loads shouldered by faculty of color—are also facets of the “whole” faculty experience to which CTL may pay greater attention. It is possible that course design is more appropriate for faculty during certain phases of their careers and less appropriate at other times. Furthermore, teaching courses using innovative methods, particularly those that are time intensive, may compromise faculty members’ abilities to balance other tasks central to their professional work. For these reasons, centers are also called upon to help faculty manage expectations relative to course design efforts, among their other priorities. To this end, we are inspired by the nascent evolution of centers like our own to embrace career planning programming that helps faculty make wise choices about their time, portfolios, and associations. Already, a host of centers have begun to offer initiatives supporting faculty research, scholarship, and creative activities; career planning and professional development; and even leadership. Such sensitivity to faculty well-being, too, is central.