It is internationally well documented that many graduate students suffer from mental health issues including stress, anxiety, and depression (Evans et al., 2018; Gewin, 2012; Hyun et al., 2007; University of California, Office of the President, 2017; Yusufov et al., 2019). Graduate students are more than six times as likely as the general population to experience depression and anxiety (Evans et al., 2018). An estimated 41% of graduate students have symptoms of moderate to severe anxiety (as compared to 6% of the general population), and an estimated 39% experience severe depression (compared to 6% of the general population) (Evans et al., 2018). A failure to achieve an appropriate work-life balance reduces well-being and quality of life (Gröpel & Kuhl, 2009; Haar et al., 2014) and has been recognized as a contributing factor to the mental health crisis among graduate students (Evans et al., 2018). Given the long hours and high-stress environments, it is not surprising that graduate students struggle with achieving work-life balance (Evans et al., 2018; University of California, Office of the President, 2017). In addition to challenges with balancing responsibilities, a majority of graduate students experiencing anxiety and depression do not report feeling valued or supported by their advisors nor do they see their advisors as being an asset to their careers (Evans et al., 2018).
To address the growing mental health crisis among graduate students and to increase retention, researchers suggest expanding professional development opportunities (Evans et al., 2018). A majority of graduate students cite a lack of institutional support for teaching-related professional development opportunities in particular (Goodwin et al., 2018), and many institutions do not require formal teaching training (Rushin et al., 1997). A US-based survey regarding teaching assistant (TA) training indicates that the majority of institutions require no formal training, with the next most common expectations being a pre-academic-year workshop, a semester-long teaching seminar, a formal teaching course, and training by a professor (Gardner & Jones, 2011; Rushin et al., 1997). The lack of pedagogy-centered career development is problematic considering graduate students need to be prepared for future employment, which may include teaching competency and communication skills (Austin & Wulff, 2004; Roberts, 2002). These are transferable skills that are internationally recognized as needing improvement in graduate education (Adams, 2002; Roberts, 2002).
Although limited pedagogical career development opportunities exist, these programs have the potential to greatly benefit graduate students. For example, reflection-based assessment for a weekly pedagogy seminar increases learning gains in questioning traditional practices, creating a statement of teaching philosophy, and maximizing student learning as well as increasing job placement success (Schram et al., 2017; Schussler et al., 2008). In addition to benefiting skills in pedagogy, instructional training provides opportunities for graduate students to participate in a learning community, which may increase sense of belonging and thus benefit mental health (Johnson et al., 2020; Liu et al., 2007). Participating in a teaching-focused community also increases self-efficacy, teaching practices, and public speaking, especially for international graduate students (Boman, 2013; Chadha, 2015; Connolly et al., 2018; Winter et al., 2015). Furthermore, graduate students learning about evidence-based pedagogies often experience synergy with research; teaching makes them better able to communicate about research and correlates with an increased number of publications relative to non-participants (Shortlidge & Eddy, 2018).
Taken together, it is clear that participating in a teaching-focused community has many benefits for mental health, sense of belonging, and career prospects. Yet the majority of institutions do not build this into the graduate student experience. Using two of our programs at the University of California, Irvine (UCI) as a case study, a quarter-long Preparing Teaching Excellence course and a year-long Pedagogical Fellows Program, we evaluated the following research questions through a mixed methods approach:
Does participation in our programs lead to increased pedagogical knowledge?
How do our pedagogy programs impact graduate students’ perceived quality of life and sense of community?
Methods
Institutional Context
UCI is a Hispanic-Serving Institution and an Asian American and Native American Pacific Islander-Serving R1 institution with roughly 30,000 undergraduate students and 6,000 graduate students enrolled and is member of the national Center for the Integration of Research, Teaching, and Learning (CIRTL) Network. UCI has a Center for Teaching and Learning that offers a Certificate in Teaching Excellence Program. By participating in the Developing Teaching Excellence course and/or the Pedagogical Fellows Program, participants will be making progress toward earning the teaching certificate.
Framework for the Developing Teaching Excellence Course
The Division of Teaching Excellence and Innovation (DTEI) has developed a quarter-long graduate student interdisciplinary Developing Teaching Excellence course (University Studies 390X or 390X) that meets for three hours a week with a maximum enrollment size of 25. The goals of the course are to:
discover, analyze, and discuss pedagogical research
identify, evaluate, and develop effective teaching practices
practice peer review of teaching
participate in a learning community.
Specific topics include icebreakers, neurobiology of learning and memory, course design and lesson planning, active learning, instructional technology, inclusive teaching, collaborative learning, and assessment. Students mainly interact with the course material through the campus learning management system. Process documents include weekly reflection papers, a diversity statement, syllabus, and a teaching-focused blog post.
To prepare for each class, students read assigned readings and write reflections. In addition to summarizing key points, students describe their thoughts and reactions and indicate how the concepts could be applied to their teaching. Reading reflection prompts also prepare students for in-class activities as well as provide a model for encouraging their own students to engage with course readings in more meaningful ways. Supporting a model of co-production of knowledge and resources, in the latter half of the course, students select their own primary research articles to read and discuss.
In class, students are placed in discipline-specific groups of four to six for the duration of the course, with opportunities to interact with members of other groups during food sharing, breaks, and observations of teaching or through learning activities such as the jigsaw or gallery walk. Everyone is given a name tent to decorate, on which they write their name, department, and pronouns. Icebreakers are facilitated throughout the course both within and across discipline groups. At the beginning of every class, the facilitator will play music from a list of songs curated by the students. In addition, groups bring food on a rotating basis, considering any necessary dietary restrictions. The assigned group describes the dishes (e.g., whether items were homemade or purchased locally from a favorite vendor). Providing food is optional—for groups not able to share for any reason, the course facilitator brought and presented food instead.
Class time follows a general structure in which the facilitator first asks if anyone has announcements or news to share with the rest of the group, followed by a warm-up discussion or activity related to the topic of the day. The facilitator varies the learning activities to demonstrate how to set up and successfully run them, followed by a debrief on the activities themselves. Groups discuss their reflection papers together while the facilitator walks around and engages with the different groups. Afterward, a whole class debrief is conducted, followed by a short discussion about additional prompts. Then, the facilitator presents new material by using short, active discussions, modeling evidence-based teaching practices (e.g., jigsaw activities) and sharing anecdotes and examples. Students are given opportunities to ask and answer questions following each exercise.
The last 15 to 20 minutes of the class are set aside for students to meet with one another regarding peer observations. For the peer observation component, students sign up to observe one another teaching at least once during the course. This includes both a pre- and post-observation meeting. During the pre-observation meeting, the observer goes over the following with the student instructor to be observed: class format and structure, goals and objectives, and student preparation requirements. In addition, the student instructor discusses how the session will be similar or different compared with others (e.g., will the instructor be trying new discussion techniques) and specific areas for feedback (e.g., are students given enough time for group activities).
During the class observation, the observer makes objective observations (e.g., noting instructional strategies, organization, interaction, student behavior) as well as provides comments and feedback. During the post-observation meeting, partners cover the following: if/how goals and objectives for the class were met, which strategies appeared most effective, which approaches could use improvement, and how students interacted with one another and the instructor. Pairs also discuss the specific aspects of the class for which the student instructor requests feedback.
To help students in completing the diversity statement assignment, students are given a list of questions to consider (e.g., how do you convey what a successful person looks like?) as well as links to UCI’s facts, inclusive milestones, demographic data, and annual report. The facilitator gives detailed feedback on each statement.
The blog post assignment serves as a teaching resource for others; students chose a pedagogical topic and write a 400- to 600-word post that synthesizes primary literature as well as draws from personal anecdotes, similar to Faculty Focus or Inside Higher Ed blog content. After reviewing submissions and providing feedback, the facilitator publishes posts on our Future Leaders in Pedagogy Development (FLIP’D) blog.
Framework for the Pedagogical Fellows Program
The DTEI developed a pedagogy-intensive program for interdisciplinary graduate students called the Pedagogical Fellows (PF) Program with a maximum enrollment of 30 students. The calendar year-long PF program requires students to meet for three hours a week during the academic quarters and a few times over the summer. The goals of the program are to:
develop and refine advanced pedagogical skills for use in higher education,
create and implement a discipline-specific TA Professional Development Program (TAPDP) for new graduate student TAs,
foster an interdisciplinary community of graduate students focused on improving pedagogical practices, and
prepare competitive application materials for the academic job market.
To apply, graduate students must have met certain prerequisites demonstrating knowledge of basic pedagogy skills. These prerequisites include either completing 390X, our Certificate in Teaching Excellence Program, or an equivalent course offered by their departments approved by the program facilitator. Among other qualities, ideal candidates have extensive teaching experience and excellent teaching evaluations that indicate continued improvement over time. The application process includes submitting teaching evaluations, a cover letter, a curriculum vitae, a letter of recommendation, and teaching-related reflections. Qualified candidates are interviewed by two current PFs as well as the Director of Graduate Student and Post-doctoral Development or the postdoc of the DTEI. The current PFs are trained in how to conduct an academic interview beforehand, and this interview served as practice for them as they entered the academic job market.
Once PFs are selected, they participate in three courses. In the first two courses, PFs read both general and discipline-specific pedagogical theory, observe and implement evidence-based teaching practices, and build community with their cohort. In addition, they apply pedagogical theory to the design of workshops for the 12-hour TAPDP, where they train incoming TAs in a variety of topics, such as lesson planning, active learning, grading, office hours, diversity and inclusion, and TA roles and responsibilities. After extensive peer feedback, the director and postdoc of the DTEI provide feedback on a complete draft of the lesson plans, presentations, and materials for the program. Before fall quarter starts, PFs implement their TAPDP as an orientation to TAs in their or a related discipline. The final course in the program focuses on academic job preparation and special topics chosen by PFs. PFs also serve as the search and interview committees for the next round of PF applicants. Facilitation of these courses is similar to what was previously described for the 390X course.
Data Collection
We collected survey data for three cohorts of 390X class participants (n = 48) as well as two cohorts of Pedagogical Fellows (n = 53). Participants completed surveys before and after completing the 390X course or the PF Program. Surveys were developed utilizing a combination of established questionnaires, inventories, and content-based assessment (Diener et al., 1985; Eaton et al., 2004; Wieman & Gilbert, 2014). For purposes of comparing our participants with the general graduate student population, we included questions that the UCI Graduate Division used to prepare the Graduate Well-Being Survey Report (University of California, Office of the President, 2017). These discrete questions drew from the Quality of Life (QoL) survey, the Center for Epidemiologic Studies Depression Scale Revised (CESD-R), and Sense of Community (SoC) questionnaire and operationalized indicators of mental health, sense of community, and quality of life. To assess whether the 390X course and the PF Program influence pedagogical knowledge, surveys included multiple choice questions assessing knowledge of module content (Appendices A–D) with performance represented as percentage of correct answers. To assess perceived quality of life, surveys also included the following open-ended question: How has participating in the 390X course or Pedagogical Fellows Program affected your quality of life?
Prior to being administered, surveys were reviewed by the UCI Center for Assessment and Applied Research and attendees at the 2018 Association for the Assessment of Learning in Higher Education conference. All surveys were approved by the UCI Institutional Review Board.
Analyses
We conducted a content analysis of the textual data collected from open-ended responses to evaluate perceived benefits in quality of life. Using open coding methods (i.e., a systematic process of coding text and identifying themes or patterns across data sets; Hsieh & Shannon, 2005), we organized codes categorically into primary and sub-themes.
To assess whether our programs met the goal of increasing pedagogical knowledge, pre-post content module analysis focused on the neurobiology of learning and memory, active learning, inclusive teaching, group work and collaborative learning, instructional technology, assessment, course design, and student learning outcomes.
For responses to the Likert scale and content questions, statistical analysis was performed using SPSS (Version 26). When comparing pre-post responses with Graduate Division data, a one-way ANOVA was performed. When comparing 390X and PF pre-post responses, a one-way ANOVA with Tukey’s post hoc test was performed. When normality and/or equal variance tests failed, analysis was done on ranks. The significance threshold was p < 0.05.
Results
Participating in 390X Improves Pedagogical Knowledge
Participants in the 390X course increased in performance in the following content modules: neurobiology of learning and memory, inclusive teaching, assessment, course design, and student learning outcomes (pre-390X vs. post-390X one-way ANOVA with Tukey’s post hoc test, p < 0.05).
PFs did not exhibit the same pedagogical knowledge gains (all modules pre-PF vs. post-PF p > 0.05), which we attribute to the requirement that they have basic pedagogical training before applying to the PF Program. However, the PFs started off with greater knowledge than those beginning the 390X course in neurobiology of learning and memory, inclusive teaching, and assessment (pre-390X vs. pre-PF one-way ANOVA with Tukey’s post hoc test, p < 0.05). In addition, after completing the PF Program, participants scored higher in all modules except for group work and instructional technology compared with 390X participants (pre/post-390X vs. post-PF one-way ANOVA with Tukey’s post hoc test, p < 0.05).
Content Knowledge Performance Note: Participants were assessed on content module knowledge before and after participating in our Developing Teaching Excellence course (390X) or our Pedagogical Fellows (PF) Program. Pre-390X participants did not necessarily have prior pedagogical knowledge, whereas pre-PF participants were required to complete pedagogical training before applying to the program. Performance is expressed as percentage correct. See Appendices A–D for details about the questions. *p < 0.05; pre-390X, n = 48; post-390X, n = 46; pre-PF, n = 19; post-PF, n = 19.
Participating in 390X and the PF Program Increases Quality of Life, Sense of Community, and Mental Well-Being
Scores on the QoL survey, CESD-R scale, and SoC questionnaire did not differ between pre- and post-program participation and compared to the general UCI graduate student population (QoL one-way ANOVA on ranks: H = 2.579, p = 0.63; CESD-R one-way ANOVA on ranks: H = 1.570, p = 0.814; 390X pre-post SoC t test: t = −1.533, p = 0.132; Table 1). When evaluating these indicators within groups, we did find statistically significant effects for “sense of community” pre- and post-PF Program (PF pre-post SoC t test: t = −2.591, p = 0.018; Table 1).
Significance Tests
| Scale | Group | Mean + SEM | p value |
|---|---|---|---|
| Quality of Life | Grad | 16.965 + 0.187 | n.s. |
| Pre-390X | 16.479 + 0.586 | ||
| Post-390X | 16.826 + 0.604 | ||
| Pre-PF | 17.263 + 0.768 | ||
| Post-PF | 16.933 + 0.802 | ||
| Center for Epidemiologic Studies Depression Scale Revised | Grad | 9.303 + 0.184 | n.s. |
| Pre-390X | 9.500 + 0.528 | ||
| Post-390X | 9.174 + 0.612 | ||
| Pre-PF | 9.842 + 0.852 | ||
| Post-PF | 9.933 + 1.304 | ||
| Sense of Community | Pre-390X | 15.500 + 0.333 | n.s. |
| Post-390X | 16.087 + 0.400 | ||
| Pre-PF | 15.737 + 0.512 | 0.018 | |
| Post-PF | 17.474 + 0.515 |
-
Note: One-way ANOVA tests were performed comparing the general graduate student population (Grad) and participants in either the Developing Teaching Excellence course (390X) or the Pedagogical Fellows (PF) Program. Max. score = 25 for each scale.
Our qualitative analysis indicates that participating in the 390X course and the PF Program benefited perceived quality of life and mental well-being. Participants see the pedagogical courses and programs as improving their quality of life by (1) promoting inclusivity (e.g., providing a welcoming space for sharing a diversity of perspectives); (2) establishing community (e.g., creating a supportive network of peers); (3) changing their perspectives (e.g., providing an opportunity for reevaluating career goals, university resources, and interests in teaching); (4) instilling confidence (e.g., improving self-confidence in teaching/mentoring students and entering the academic job market); (5) equipping them with skills and tools (e.g., thinking creatively, using new pedagogical techniques); (6) promoting positive emotional well-being (e.g., reigniting a passion for teaching, easing stress); (7) providing a break from research (e.g., allowing an opportunity to focus on other areas of academic growth); and (8) alleviating concerns related to food (e.g., providing a consistent lunch).
Unique to the PF data, in addition to providing a network of supportive peers, the pedagogical community also provided stability, friendships, and a greater sense of connection to the larger university community. PFs emphasized the importance of the stability (e.g., structured, consistent community support) provided through the three-quarter program. In addition, PFs discussed how the program improved their quality of life by providing opportunities to establish enduring friendships that supported them through times of difficulty. Furthermore, PFs saw their pedagogical community as connecting them to larger networks across the university, which in turn bolstered their self-confidence.
No students discussed these courses as having an overall negative impact on their quality of life. However, a few students mentioned challenges associated with participating in the 390X course and the PF Program. First, students occasionally had difficulties balancing coursework and associated program obligations with other responsibilities (e.g., research). Students discussed how these stressful situations were ameliorated through the understanding and support of the course facilitator and peers. Second, a few students felt that while providing opportunities to interact with a diversity of students was positive overall, these cross-disciplinary conversations occasionally led to unpleasant interactions (e.g., microaggressions). It is important to note that while these few students mentioned challenges, each emphasized that the course(s) still contributed net positive experiences to their quality of life. All of these key themes are discussed in greater detail below.
Inclusivity
Graduate students in both the 390X course and the PF Program described their respective pedagogical course(s) as being a welcoming space for students to gain exposure to other perspectives and disciplines across campus as well as a space safe for expressing their true passions for teaching.
Several students characterized the ability to learn from and interact with students from other disciplines as “refreshing” and appreciated the opportunity to be able to meet and engage with people with other backgrounds and experiences. For example, one student wrote, “It is always refreshing to learn a new discipline and work with people from other disciplines.” In addition to discussing the benefits of the cross-disciplinary interaction across schools on campus, students also appreciated the cross-departmental interaction within schools as well. For example, one student wrote, “I have learned much from my interaction from other students in the Humanities. I believe it is a practice that should be kept since many of the issues we discuss applies to our departments.”
In addition to appreciating a space welcoming of campus-wide experiences and orientations, students characterized the course(s) as specifically being an inclusive space for them personally. Also characterized as “refreshing,” students frequently described the course as providing a welcoming and safe space where they could truly be themselves and share their passions for pedagogy. For example, one student stated, “The sense of community and non-formal operation of this class made it feel welcoming.” Another student noted, “I feel as though coming to [the course] has been a safe place for me to be surrounded by people who are passionate and exceptional at whatever they choose to pursue in life, and for that I’m so grateful.”
Community
Students who participated in the 390X course and the PF Program frequently discussed how the community established through the pedagogical course(s) positively impacted their quality of life. This was the most salient theme across data sets. For both cohorts, peers were seen as supportive and unique. For the PF cohort, the community of peers was described as providing friendship and stable, emotional support.
Both the 390X and PF communities were commonly characterized as a group of like-minded, diverse, relatable, kind, and encouraging peers. Students commonly discussed the importance of being a part of a community of peers genuinely supportive of their interests in pedagogy and committed to helping one another succeed. As one student wrote, “Everyone in this class is looking for ways to help each other.” Students in both programs appreciated the opportunity to troubleshoot pedagogy-related problems and share ideas, seeing their peers as pedagogy experts that served as both a resource for teaching-based questions and as a support group. As an example, one student remarked, “I have a support group and people I could ask for help with running different classes.”
Students within both programs described their community of peers as “special” and different than other academics, due to their interests in teaching. As an example, one student stated, “It is nice to be surrounded by a supportive community where everyone is interested in teaching and isn’t looking to put anyone down for wanting a teaching position (as I can often find to be the case at discipline-specific conferences).” Furthermore, students discussed how these needed forms of support and opportunities for social engagement were more difficult to find within their respective departments. For example, one student stated, “I really feel like I built a little community with my group. I have better rapport with them than many people in my department!” Another student echoed these sentiments, writing, “My cohort is weird and toxic, so [this] is like the cohort I never had. I think there’s something special about people who are interested in pedagogy in particular—they are humble and not arrogant in the ways most academics are.”
While both the 390X and the PF cohorts frequently discussed the benefits of the community built through the course, the PFs described their community in even more positive terms. PFs often characterized their community as fostering close friendships. For example, one student stated, “I’ve loved making friends with new people in the program and learning from a diverse community of fellow teachers.” As another example, one student wrote, “I have loved making new friends in the PF program. This improves my quality of life.”
In addition, PFs (who met on the same day and at the same time for three quarters in a row) felt the program improved their quality of life by providing needed stability and enduring friendships. These students felt that structured and reliable support and community networks kept them optimistic and helped them through personal difficulties and challenges. As an example, one student wrote, “It has been wonderful to have this space in my schedule over the past year, even as other aspects of my academic life have been somewhat stressful.” Many PFs stated that the consistent friendships improved their quality of life and provided them the social and emotional support needed in difficult times. As an example, a student stated:
The PF Program has vastly improved my quality of life by providing me pedagogical support and social engagement outside of my department. I’ve met some of my closest friends through this program, but aside from that I looked forward to meeting every week because so often I didn’t have the time or opportunities to socialize with peers outside of these Tuesday meetings. This group kept me sane, optimistic, well-fed, and engaged and overall helped me in ways that I think have reverberated throughout many of the different aspects of my life. I’ve had an extremely difficult year personally, otherwise, and I do think that the PF Program helped me weather those difficulties in incalculable and unexpected ways by providing a sort of stability I haven’t found elsewhere.
Another student echoed these sentiments, stating, “Going to [PF] classes each week has honestly been the highlight of my week at some of the hardest times in my life.”
Finally, PFs felt more connected to the larger university community through their network of multidisciplinary peers. Furthermore, these ties were seen as advancing professional opportunities and working relationships as well as boosting personal confidence at university-wide events and spaces. As an example, one student commented on the professional benefits stating:
The network we’ve built is great—I’m part of a school-specific initiative to deliver some workshops within [our school], and they recruited several PFs to do this. It’s been very easy to put all that together, since we’ve worked closely with each other for the best part of a year now.
Reflecting on the larger university network, another PF wrote, “I also really enjoyed being able to find people I knew (PFs) at almost every event on campus, which made me feel more comfortable and confident at UCI.”
Perspectives
Both cohorts discussed how the 390X and PF courses improved their quality of life by changing their perspectives on the university, their experiences/purposes in graduate school, and their career goals and opportunities.
Students from both cohorts discussed how the pedagogical course(s) and programs changed their view of the university. Students perceived the university more positively, due to the support for teaching. Students discussed how they were previously unaware of the resources and support for teaching on campus; importantly, these characteristics made the university seem positively unique from other institutions. For example, one student stated:
[M]y perspective on UCI has changed. Before this class started, I was committed to transferring out of UCI by the end of the academic year. Now I’m a little more undecided because I recognize this great teaching community around me. I’m not sure I could find that in other places.
As another example, one student reflected that the pedagogical course(s) “made me value the university environment.”
For some, this change in perspective was framed as important in determining whether to continue graduate school at the university or whether to continue graduate school at all. For others, these courses helped them identify how they were unique as graduate students and helped remind them why they had decided to pursue graduate school to begin with. As one student wrote:
I think that participating in 390X has positively affected my quality of life. It has made me more motivated to stay in graduate school in order to pursue the teaching career I want with all of the awesome teaching resources that DTEI has to offer. Before enrolling in this class, I was pretty annoyed at myself for staying in grad school and spending so much time on research, which I don’t want to do as a career. After taking this class, I am grateful for being pushed to guest lecture a class for my PI who just wants me in the lab all the time, and realizing that I actually do like teaching, and I could potentially be good at it and reach so many people! I am now pursuing more TAships to prepare for a future career in teaching and have found more meaning in staying in graduate school in general. So, long story short, I feel happier and more purposeful now.
Another student shared a similar change in perspective, stating, “Being part of the PF Program has definitely been a big reminder of why I even came to graduate school in the first place.”
Students from both cohorts reflected on how the pedagogy course(s) changed their career goals and opened up additional opportunities for their academic futures. These students commonly discussed how the courses either made them realize how much they enjoyed teaching and/or reignited their passions for teaching. Relatedly, these courses and their community of peers introduced, legitimated, and/or reinforced the possibility of teaching as a career path. Importantly, several students realized that teaching would likely be a more fulfilling career and felt more hopeful for their futures. As an example, one student wrote that “[the program] has also improved my quality of life by helping me learn about various opportunities that may lead to a more fulfilling career than if I hadn’t been exposed to facets of education research.”
Confidence
Another highly salient theme across the cohorts, students commonly discussed how the course(s) had improved their confidence in a myriad of ways: as a teacher, mentor, and professional and as a graduate student on the job market. Furthermore, the courses promoted general self-confidence, with students discussing their appreciation for their unique, creative selves. Students discussed these improvements in confidence as making them more hopeful about their futures.
Most commonly, students discussed the pedagogy course(s) as improving their confidence in their abilities to teach and mentor effectively, to design a course, and to speak in front of the class. Many students described previous feelings of anxiety and “impostor syndrome” related to serving as a TA at the university or as a professor after graduating. For example, one student wrote:
I feel so much more confident as a leader and a mentor after the PF Program. Although I’ve been teaching for a long time, taking the PF Program while designing my first course made me feel like I could actually do it (rather than suffering from horrible impostor syndrome).
Another commented, “I’m less stressed and anxious about teaching. Improving my confidence in my ability to teach (and providing me with a wide range of strategies and feedback) has been the most valuable part of this course.” Students discussed how greater self-confidence in leading a class effectively, engaging students, and designing a course helped to alleviate these fears and make teaching more enjoyable. For example, one student stated, “[The class] has made me more confident about teaching and as a job candidate. It has made me excited about designing classes and about thinking about my teaching plans.” Furthermore, students felt more confident as pedagogical experts, by helping others with teaching and providing feedback for peers by observing their teaching. For example, one student remarked, “I also feel comfortable mentoring others about teaching.” In addition, students in both cohorts discussed how the course(s) contributed to greater confidence in their professional capabilities, especially in terms of gaining confidence in communication and public speaking skills. For example, one student wrote that “the program has helped me become more confident with my teaching and public speaking skills, so the PF Program has been an overall positive experience in my life.”
Students commonly stated that the pedagogical course(s) made them feel more confident about their prospects on the job market. Relatedly, students also felt more confident about attaining their long-term career goals and their abilities to be successful in their careers. To this point, one student wrote:
Prior to this class I was unsure about how to structure a class of my own. Having learned about active learning, inclusivity, and curriculum development I feel more at ease about going on the job market in the near future and well-equipped to land a teaching-heavy job if that is what I wish.
Similarly, another student wrote that the course “helped me to understand what I want to do after I finish my PhD and gave me confidence that I’ll be good at it.”
Beyond teaching skills, professional skills, and career goals, students expressed having greater self-confidence in general. As an example, one student commented that “[t]his program has been so important in improving my self-confidence.” This improved general self-confidence among students related to a positive reflection of their unique abilities and a value for their skills and perspectives. As an example, students commented on feeling more creative and capable as a student and as a researcher. One student wrote:
While I have learned a lot about teaching, the most valuable thing [the courses] have helped me learn is to value my own (and others’) unique skills, interests, and approaches to teaching and learning. This self-confidence has made me a more curious, creative, and happy student, teacher, and researcher.
Skills and Tools
Students in both cohorts wrote that the pedagogy course(s) equipped them with useful skills and tools. Most commonly, students reflected on being more skilled at teaching and more equipped to apply for jobs and excel as an academic professional. For example, one student wrote:
This class has equipped me with pedagogical tools to enhance my pedagogy and has helped me feel prepared to step in front of a classroom. Overall, I am glad I took this course and appreciate the tools it gave me to be a better instructor.
As another example, one student stated that “[the course] has given me a good ideal of the responsibility of being an effective TA and the skills, resources that are important in being an instructor.” Having these skills/tools was characterized as reducing the stress and fears associated with teaching. As an example, one student stated, “I have learned and, most important, practiced pedagogical techniques that have given me more security in my TA practice.”
Other more generalized skills were commonly mentioned as well. For example, students felt the course(s) helped them to think more creatively and “outside of the box” and provided invaluable knowledge; these widely applicable skills and tools were described as being useful in multiple facets of life, from conducting research to parenting. As one student wrote:
I have learned so much in the [course(s)], not just as a teacher, but other wide-ranging critical skills that apply to my life as a researcher, facilitator and overall professional. I also think the creative [pedagogy] knowledge and skills I have learned can apply to other areas of my life, like how to better teach my own daughter and encourage her to be a life-long learner.
Another wrote that “390X made me think outside the box a lot, and that is invaluable!”
Positive Emotional Well-Being
Across both data sets, students frequently discussed how the pedagogical course(s) gave them a sense of positive well-being. As an example, one student stated, “I feel relaxed and happy whenever I am in this class.” Many described the class(es) as an enjoyable, engaging, low-stress way for communicating with peers and skill building. For example, one student stated, “I enjoy laughing and smiling with everyone.” Another said, “[I] love the low-stress environment and food which made the class enjoyable, and I looked forward to attending each week.” Students across cohorts expressed that they looked forward to attending class and commonly described the class as the highlight of their week. For example, one student commented, “I look forward to Tuesdays every week! It’s a highlight.” Several PFs stated they were grateful for the experience of participating in the program, describing the PF Program as their best experience of graduate school. For example, one student stated:
All of this, from the positive social interactions to the skills and confidence gained, have improved my overall happiness and well-being as a person. This program has certainly been the best experience of my graduate career, and I’m so glad I was able to be a part of it.
Research Break
Students in both the 390X course and the PF Program discussed how the pedagogical classes offered a welcomed break from research. Some students described their research as overwhelming, stressful, and isolating. For example, one student wrote, “Getting overwhelmed in research is a lot to handle, especially because it is not what I want to be doing long term.” The pedagogical courses were seen as improving quality of life by offering students a mental break from having to think about research as well as providing an opportunity to make progress in another area of their academic life. As one student reflected, “I loved participating in this class and having something else to think about for a few hours a day that is not my research.” Another student stated, “[I] felt like I was making progress in an area other than research.” As a final example, another wrote, “The dissertation phase can be especially isolating, so it is nice to look forward to being with others experiencing similar challenges.”
Food
While a less common theme, in both cohorts the students discussed benefits of having food during class. Food was characterized as a morale booster that made class more enjoyable. Furthermore, students commented on the benefits of being well fed and not having to worry about food on class days. For example, one student stated that “this class helped me not to worry about my food and to be well fed, at least for a day.”
Challenges
While all of the responses framed the course(s) as overall positive and beneficial, there were a few students who mentioned challenges associated with the classes. Students at times found balancing the course responsibilities with other duties to be difficult. However, flexibility and support by the course facilitator and peers may have ameliorated these stressors. For example, one student noted, “the homework was a little stressful to complete. It always took me twice as long because of my difficulties with English. , but [the facilitator] was very understanding and patient, which I greatly appreciate.” Another student stated:
Having the PF Program on top of all my other grad student responsibilities is an additional stress and has slowed my progress with my own research, but ultimately it is a net positive because it has allowed me to grow in an area of my work (pedagogy) that I value most and has helped me feel connected to people who share similar interest or passion in teaching.
As another example, one student noted that “at times it was hard to balance the multilayered discipline specific work, but I am so grateful for this experience and for [the facilitator’s] patience at times when the balance was less successful.” Two 390X students, however, mentioned challenges associated with their peers; one student described their departmental working groups as difficult to work with, and another student expressed experiencing microaggressions from students in other disciplines.
Discussion
We have previously reported on an effective graduate student and postdoctoral scholar professional development pipeline at the DTEI (Mann et al., 2022), which addresses the need to develop future faculty in the ever-changing higher education landscape, including developing soft skills and evidence-based teaching practices (Adams, 2002; Austin & Wulff, 2004; Connolly et al., 2018; Hill et al., 2019; Roberts, 2002). Here, we evaluated the effectiveness of two of our programs, the Developing Teaching Excellence course (390X) and the Pedagogical Fellows (PF) Program, in bolstering pedagogical knowledge. We also explored how our programs might benefit perceived quality of life and sense of community. This is especially relevant given multiple reports of mental health issues among graduate students (Evans et al., 2018; Gewin, 2012; Hyun et al., 2007; University of California, Office of the President, 2017; Yusufov et al., 2019). Evans et al. (2018) found that 40% of graduate students surveyed around the world had moderate to severe ranges of anxiety and depression and attributed the high prevalence of mental health issues to inadequate work-life balance and advisor relationships. To create better work-life balance situations for graduate students, Evans et al. suggest intervention strategies such as career professional development and cultural changes within higher education.
Although we did not see quantitative improvements in quality of life surveys, our qualitative analysis indicates that our programs have a positive impact on students, supporting the recommendation by Evans et al. (2018) that graduate career development programs should be part of the strategy in addressing the graduate student mental health crisis. Interestingly, our results indicate that there are similar gains in mental health and sense of belonging between quarter-long and year-long pedagogy programs, which may help with transferability to different institutions with different resources. Additional research investigating the connection between instructional development programs and gains in mental health outcomes should be investigated further.
One of the surprising findings was the change in perspective of UCI as a whole, specifically how participation in our programs reaffirmed the decision to finish graduate school. Given the high prevalence of students who drop out of graduate school (Cooke et al., 1995; de Valero, 2001; Most, 2008; Wollast et al., 2018), our findings provide additional evidence and support showing that social support is key for improving graduate degree completion rates (Golde, 2005; Jairam & Kahl, 2012). Notably, there was an increase in the sense of community with PFs after completing the PF Program, which could be attributed to it being a year-long cohort-based program. There was also specific mention of the 390X course in motivation for staying in graduate school. It will be interesting to gather data on graduation rates for students who completed our programs to see if there was an impact.
As mentioned previously (Mann et al., 2022), these programs are scalable for different size CTLs and programs. For institutions with little to no CTL support, the first step would be to identify someone who is aware of the pedagogical literature and has experience working in a CTL. That person can then develop programs targeting graduate students and even postdoctoral scholars and faculty. Creating partnerships with liaisons in the different schools and departments helps ensure that faculty are aware of the programs and the benefits to graduate student professional development. If available, we suggest working closely with the institution’s graduate division to both complement current offerings and use the graduate division’s resources and reach to help advertise professional development programs. Our strategy has been to use a conveyor belt model, leaning heavily on former 390X and PF Program participants to provide feedback and insight into improvements and facilitate additional programming, which is how the 390X course has integrated CIRTL values, diversity statements, and blog posts to help participants further engage with one another and with the DTEI.
In conclusion, graduate students need better emotional, social, and professional support in their programs in order to be successful in both academics and in the job market. Although different institutions have different professional development programs, they have been sparsely studied in a data-driven approach, especially for their potential impact in quality of life and sense of community improvement. Formal pedagogical training through our 390X course and PF Program teaches transferable soft skills and self-efficacy and provides a common space for people who care about teaching to meet and learn together in an interdisciplinary manner. In addition, participating in professional development opportunities synergizes with research obligations and can lead to a more balanced lifestyle. This could improve quality of life and mental health in a vulnerable academic population.
Biographies
Daniel Mann is the Director of Graduate Student and Postdoctoral Scholar Instructional Development at the Division of Teaching Excellence and Innovation (DTEI) at the University of California, Irvine. Daniel holds a Ph.D. in Cognitive Science, and his research interests focus on assessment of educational development, graduate student development, and the cognitive science of learning.
Ashley Hooper is a Postdoctoral Scholar with the Division of Teaching Excellence and Innovation (DTEI) at the University of California, Irvine. In her current position, she teaches graduate level pedagogy courses and supports doctoral students and postdoctoral fellows in their career development. Her current research interests focus on the potential for pedagogy training in alleviating ‘imposter syndrome’ and fostering supportive academic peer groups. She holds a Ph.D. in Urban and Environmental Planning and Policy.
Matthew Mahavongtrakul is a Postdoctoral Scholar with the Division of Teaching Excellence and Innovation (DTEI) at the University of California, Irvine. He was the facilitator for the Developing Teaching Excellence and Advanced Course Design courses and currently provides formal pedagogical training to graduate students, post-doctoral scholars, and faculty. Matthew holds a Ph.D. in Biological Sciences.
Brian Sato is a Professor of Teaching of Molecular Biology and Biochemistry and Associate Dean for the Division of Teaching Excellence and Innovation at the University of California Irvine. In this role, he oversees the campus’ professional development activities for faculty and future faculty and conducts STEM education research to create more inclusive classroom environments.
References
Adams, K. A. (2002). What colleges and universities want in new faculty. Preparing Future Faculty Occasional Paper Series.
Austin, A. E., & Wulff, D. H. (2004). The challenge to prepare the next generation of faculty. In Wulff, D. H. & Austin, A. E. (Eds.), Paths to the professoriate: Strategies for enriching the preparation of future faculty (pp. 3–16). Jossey-Bass.
Boman, J. S. (2013). Graduate student teaching development: Evaluating the effectiveness of training in relation to graduate student characteristics. Canadian Journal of Higher Education, 43(1), 100–114.
Chadha, D. (2015). Evaluating the impact of the graduate certificate in academic practice (GCAP) programme. International Journal for Academic Development, 20(1), 46–57. https://doi.org/10.1080/1360144X.2014.940956https://doi.org/10.1080/1360144X.2014.940956
Connolly, M. R., Lee, Y.-G., & Savoy, J. N. (2018). The effects of doctoral teaching development on early-career STEM scholars’ college teaching self-efficacy. CBE—Life Sciences Education, 17(1), Article 14. https://doi.org/10.1187/cbe.17-02-0039https://doi.org/10.1187/cbe.17-02-0039
Cooke, D. K., Sims, R. L., & Peyrefitte, J. (1995). The relationship between graduate student attitudes and attrition. The Journal of Psychology, 129(6), 677–688. https://doi.org/10.1080/00223980.1995.9914938https://doi.org/10.1080/00223980.1995.9914938
de Valero, Y. F. (2001). Departmental factors affecting time-to-degree and completion rates of doctoral students at one land-grant research institution. The Journal of Higher Education, 72(3), 341–367. https://doi.org/10.2307/2649335https://doi.org/10.2307/2649335
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The satisfaction with life scale. Journal of Personality Assessment, 49(1), 71–75. https://doi.org/10.1207/s15327752jpa4901_13https://doi.org/10.1207/s15327752jpa4901_13
Eaton, W. W., Smith, C., Ybarra, M., Muntaner, C., & Tien, A. (2004). Center for Epidemiologic Studies Depression Scale: Review and revision (CESD and CESD-R). In Maruish, M. E. (Ed.), The use of psychological testing for treatment planning and outcomes assessment: Vol. 3. Instruments for adults (pp. 363–377). Lawrence Erlbaum Associates Publishers.
Evans, T. M., Bira, L., Gastelum, J. B., Weiss, L. T., & Vanderford, N. L. (2018). Evidence for a mental health crisis in graduate education. Nature Biotechnology, 36(3), 282–284. https://doi.org/10.1038/nbt.4089https://doi.org/10.1038/nbt.4089
Gardner, G. E., & Jones, M. G. (2011). Pedagogical preparation of the science graduate teaching assistant: Challenges and implications. Science Educator, 20(2), 31–41.
Gewin, V. (2012). Mental health: Under a cloud. Nature, 490(7419), 299–301. https://doi.org/10.1038/nj7419-299ahttps://doi.org/10.1038/nj7419-299a
Golde, C. M. (2005). The role of the department and discipline in doctoral student attrition: Lessons from four departments. The Journal of Higher Education, 76(6), 669–700. https://doi.org/10.1353/jhe.2005.0039https://doi.org/10.1353/jhe.2005.0039
Goodwin, E. C., Cao, J. N., Fletcher, M., Flaiban, J. L., & Shortlidge, E. E. (2018). Catching the wave: Are biology graduate students on board with evidence-based teaching? CBE—Life Sciences Education, 17(3), Article 43. https://doi.org/10.1187/cbe.17-12-0281https://doi.org/10.1187/cbe.17-12-0281
Gröpel, P., & Kuhl, J. (2009). Work–life balance and subjective well-being: The mediating role of need fulfilment. British Journal of Psychology, 100(2), 365–375. https://doi.org/10.1348/000712608×337797https://doi.org/10.1348/000712608×337797
Haar, J. M., Russo, M., Suñe, A., & Ollier-Malaterre, A. (2014). Outcomes of work–life balance on job satisfaction, life satisfaction and mental health: A study across seven cultures. Journal of Vocational Behavior, 85(3), 361–373. https://doi.org/10.1016/j.jvb.2014.08.010https://doi.org/10.1016/j.jvb.2014.08.010
Hill, L. B., Austin, A. E., Bantawa, B., & Savoy, J. N. (2019). Factors of success: Building and sustaining teaching professional development opportunities for doctoral students and postdocs. Higher Education Research & Development, 38(6), 1168–1182. https://doi.org/10.1080/07294360.2019.1616677https://doi.org/10.1080/07294360.2019.1616677
Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687https://doi.org/10.1177/1049732305276687
Hyun, J., Quinn, B., Madon, T., & Lustig, S. (2007). Mental health need, awareness, and use of counseling services among international graduate students. Journal of American College Health, 56(2), 109–118. https://doi.org/10.3200/jach.56.2.109-118https://doi.org/10.3200/jach.56.2.109-118
Jairam, D., & Kahl, D. H. (2012). Navigating the doctoral experience: The role of social support in successful degree completion. International Journal of Doctoral Studies, 7(31), 311–329. https://doi.org/10.28945/1700https://doi.org/10.28945/1700
Johnson, M. D., Sprowles, A. E., Goldenberg, K. R., Margell, S. T., & Castellino, L. (2020). Effect of a place-based learning community on belonging, persistence, and equity gaps for first-year STEM students. Innovative Higher Education, 45(6), 509–531. https://doi.org/10.1007/s10755-020-09519-5https://doi.org/10.1007/s10755-020-09519-5
Liu, X., Magjuka, R. J., Bonk, C. J., & Lee, S. h. (2007). Does sense of community matter? An examination of participants’ perceptions of building learning communities in online courses. Quarterly Review of Distance Education, 8(1), 9–24, 87–88.
Mann, D., Mahavongtrakul, M., & Hooper, A. (2022). An efficient and coherent pipeline for graduate student and postdoctoral scholar educational development. To Improve the Academy, 41(1).
Most, D. E. (2008). Patterns of doctoral student degree completion: A longitudinal analysis. Journal of College Student Retention: Research, Theory & Practice, 10(2), 171–190. https://doi.org/10.2190/CS.10.2.dhttps://doi.org/10.2190/CS.10.2.d
Roberts, G. (2002). SET for success: The supply of people with science, technology, engineering and mathematics skills. HM Treasury London.
Rushin, J. W., De Saix, J., Lumsden, A., Streubel, D. P., Summers, G., & Bernson, C. (1997). Graduate teaching assistant training: A basis for improvement of college biology teaching & faculty development? The American Biology Teacher, 59(2), 86–90. https://doi.org/10.2307/4450255https://doi.org/10.2307/4450255
Schram, L. N., Pinder-Grover, T., & Turcic, S.. (2017). Assessing the long-term impact of the Preparing Future Faculty seminar. To Improve the Academy, 36(2), 101–116. https://doi.org/10.1002/tia2.20063https://doi.org/10.1002/tia2.20063
Schussler, E., Torres, L. E., Rybczynski, S., Gerald, G. W., Monroe, E., Sarkar, P., Shahi, D., & Osman, M. A. (2008). Transforming the teaching of science graduate students through reflection. Journal of College Science Teaching, 38(1), 32–36.
Shortlidge, E. E., & Eddy, S. L. (2018). The trade-off between graduate student research and teaching: A myth? PLoS One, 13(6), Article e0199576. https://doi.org/10.1371/journal.pone.0199576https://doi.org/10.1371/journal.pone.0199576
University of California, Office of the President. (2017). Graduate student well-being survey report. https://ucop.edu/institutional-research-academic-planning/_files/graduate_well_being_survey_report.pdfhttps://ucop.edu/institutional-research-academic-planning/_files/graduate_well_being_survey_report.pdf
Wieman, C., & Gilbert, S. (2014). The teaching practices inventory: A new tool for characterizing college and university teaching in mathematics and science. CBE—Life Sciences Education, 13(3), 552–569. https://doi.org/10.1187/cbe.14-02-0023https://doi.org/10.1187/cbe.14-02-0023
Winter, J., Turner, R., Gedye, S., Nash, P., & Grant, V. (2015). Graduate teaching assistants: Responding to the challenges of internationalisation. International Journal for Academic Development, 20(1), 33–45. https://doi.org/10.1080/1360144X.2014.986661https://doi.org/10.1080/1360144X.2014.986661
Wollast, R., Boudrenghien, G., Van der Linden, N., Galand, B., Roland, N., Devos, C., de Clercq, M., Klein, O., Azzi, A., & Frenay, M. (2018). Who are the doctoral students who drop out? Factors associated with the rate of doctoral degree completion in universities. International Journal of Higher Education, 7(4), 143–156. https://doi.org/10.5430/ijhe.v7n4p143https://doi.org/10.5430/ijhe.v7n4p143
Yusufov, M., Nicoloro-SantaBarbara, J., Grey, N. E., Moyer, A., & Lobel, M. (2019). Meta-analytic evaluation of stress reduction interventions for undergraduate and graduate students. International Journal of Stress Management, 26(2), 132–145. https://psycnet.apa.org/doi/10.1037/str0000099https://psycnet.apa.org/doi/10.1037/str0000099
Appendix A. Pre-390X Survey
Would you like to opt out of including your responses in our analysis?
Yes
No
Rank the value of the following aspects of your motivation for taking this class. Please select only one response per column (1 = Least important; 2 = Somewhat important; 3 = Important; 4 = Very important; 5 = Most important):
Improving quality of life
Preparing job market materials
Developing teaching skills
Having a sense of community
Building a network with PFs and the DTEI
For the following statements, click the checkbox if the statement is true.
We mostly only use 10% of our brain.
Students’ prior knowledge can help or hinder their learning.
Visual learners always learn best when they receive information visually.
Right- or left-brain dominance helps explain individual differences among learners.
Once synapses are established, they do not change.
Only the hippocampus is necessary when recalling a memory.
Learning is hindered when cognitive load is maxed.
There is evidence that effectively using active learning increases student retention of knowledge.
Using ice-breakers guarantees more participation.
Cell phones in class are ineffective active learning tools.
A student reading a passage in class is not an example of active learning.
A student’s cultural background influences learning.
Using a specific pop cultural reference is a good way to increase inclusivity.
Since course content is standardized for all students in a given course, diversity does not play a role in content delivery.
Individual students’ learning styles should govern the way material is learned.
None of the above statements are true.
Which of the following is the most likely way to engage introverted students in the classroom?
Doing student presentations
Using a reflective technique, such as a minute paper
Holding a debate
Randomly calling on students to answer questions
An educational framework that addresses individual learning differences in the classroom by encouraging instructors to implement a flexible and accessible learning environment is called:
Diversity
Collaborative learning
Universal design for learning
Open educational resource
Which of the following is an effective way of managing groups?
Only having group assessments
Having students self-select their groups
Having groups meet solely outside of class to work on their project
Implementing peer feedback as a form of assessment
When designing an activity centered on instructional technology, what is the most important thing an instructor should consider?
Student engagement with the activity
Feasibility of the activity
Alignment of the activity to the learning objectives
Time required to do the activity
Which of the following is the best example of summative assessment?
Standardized exams (MCAT, GRE, GMAT, DAT, PCAT, etc.)
Weekly quizzes
Think-Pair-Share
Pre-reading worksheets
Which of the following is the best example of formative assessment?
A cumulative final exam
A midterm exam covering one entire unit of a class
A concept check quiz given intermittently throughout a unit that isn’t worth any points
A critique paper addressing all of the readings in a class
Backward course design begins with
Choosing the readings
Getting materials from the former professor
Establishing learning goals and objectives
Selecting teaching techniques
This is a quality learning objective: “Students will be able to understand experimental psychology.”
True
False
Have you taught a course, either as an instructor or TA, over the past year? If you did not instruct students in the classroom over the past year (i.e., you had an administrative role), please select “No.”
Yes
No
What department(s) was/were the class(es) you taught in over the past year?
Which of the following pedagogical topics have you used (or have not used, but plan to) in your classrooms over the past year in a uniersity setting? In what capacity? If you have not used the topic, please type “N/A.”
Neuroscience of learning and memory
Active learning
Inclusive teaching
Collaborative learning
Instructional technology
Formative assessment
Summative assessment
Backward design
In a typical class session, what is the average percentage of class (0–20%, 21–40%, 41–60%, 61–80%, 81–100%) that you:
pause to ask for questions?
have small group discussions or problem-solving?
show demonstrations, simulations, or video clips?
Over the course of a term for one class, how many times do you have discussions on why the material is useful and/or interesting from students’ perspective?
0–2 times a quarter
Every few class sessions
At least once a week
Every session
Check all that occurred in the most recent course you taught:
Assessment of background knowledge
Students were asked to read/view material for upcoming class session
Students were asked to read/view material for upcoming class session and complete assignments or quizzes on it shortly before class or at beginning of class
Reflective activity at end of class, e.g., “one-minute paper” or similar (students briefly answering questions, reflecting on lecture and/or their learning, etc.)
Student presentations (verbal or poster)
Tried new pedagogical techniques (e.g., active learning, collaborative learning, inclusive strategies, instructional technology)
None of the above
What fraction of a typical class period do you spend lecturing (presenting content, deriving mathematical results, presenting a problem solution, etc.)?
A. 0–20%
B. 21–40%
C. 41–60%
D. 61–80%
E. 81–100%
During your graduate and/or postdoc career, how much exposure have you had to an interdisciplinary community anywhere on campus? “Interdisciplinary” in this context means people from different schools on campus. For example, if you are a student in the School of Biological Sciences, how much interaction have you had with a student in the School of Education?
No exposure
Little exposure
Some exposure
A lot of exposure
Everyday exposure
On a scale of 1–5, rate the value of (1 - Not at all valuable; 2 - Slightly valuable; 3 - Somewhat valuable; 4 - Very valuable; 5 - Extremely valuable):
Learning from an interdisciplinary group
Having a supportive group with teaching as a common passion
Having a supportive group with research as a common passion
Doing activities (social, professional development, etc.) outside of your department
Opinions of people in different disciplines about your teaching
Opinions of people in your discipline about your teaching
Opinions of people in different disciplines about your research
Opinions of people in your discipline about your research
Do you have any other comments about participating in an interdisciplinary community?
On a scale of 1–5, please indicate the frequency of the following (1 - Not at all or less than one day last week; 2 - One or two days last week; 3 - Three to four days last week; 4 - Five to seven days last week; 5 - Nearly every day for two weeks):
My appetite was poor.
My sleep was restless.
I felt sad.
I lost interest in my usual activities.
I could not focus on the important things.
On a scale of 1–5, please rate your level of agreement (1 - Strongly disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Strongly agree).
In most ways my life is close to my ideal.
The conditions of my life are excellent.
I am satisfied with life.
So far I have gotten the important things I want in life.
If I could live my life over, I would change almost nothing.
Appendix B. Post-390X Survey
Would you like to opt out of including your responses in our analysis?
Yes
No
Rank the value of the following aspects of your motivation for taking this class. Please select only one response per column (1 = Least important; 2 = Somewhat important; 3 = Important; 4 = Very important; 5 = Most important):
Improving quality of life
Preparing job market materials
Developing teaching skills
Having a sense of community
Building a network with PFs and the DTEI
For the following statements, click the checkbox if the statement is true.
We mostly only use 10% of our brain.
Students’ prior knowledge can help or hinder their learning.
Visual learners always learn best when they receive information visually.
Right- or left-brain dominance helps explain individual differences among learners.
Once synapses are established, they do not change.
Only the hippocampus is necessary when recalling a memory.
Learning is hindered when cognitive load is maxed.
There is evidence that effectively using active learning increases student retention of knowledge.
Using ice-breakers guarantees more participation.
Cell phones in class are ineffective active learning tools.
A student reading a passage in class is not an example of active learning.
A student’s cultural background influences learning.
Using a specific pop cultural reference is a good way to increase inclusivity.
Since course content is standardized for all students in a given course, diversity does not play a role in content delivery.
Individual students’ learning styles should govern the way material is learned.
None of the above statements are true.
Which of the following is the most likely way to engage introverted students in the classroom?
Doing student presentations
Using a reflective technique, such as a minute paper
Holding a debate
Randomly calling on students to answer questions
An educational framework that addresses individual learning differences in the classroom by encouraging instructors to implement a flexible and accessible learning environment is called:
Diversity
Collaborative learning
Universal design for learning
Open educational resource
Which of the following is an effective way of managing groups?
Only having group assessments
Having students self-select their groups
Having groups meet solely outside of class to work on their project
Implementing peer feedback as a form of assessment
When designing an activity centered on instructional technology, what is the most important thing an instructor should consider?
Student engagement with the activity
Feasibility of the activity
Alignment of the activity to the learning objectives
Time required to do the activity
Which of the following is the best example of summative assessment?
Standardized exams (MCAT, GRE, GMAT, DAT, PCAT, etc.)
Weekly quizzes
Think-Pair-Share
Pre-reading worksheets
Which of the following is the best example of formative assessment?
A cumulative final exam
A midterm exam covering one entire unit of a class
A concept check quiz given intermittently throughout a unit that isn’t worth any points
A critique paper addressing all of the readings in a class
Backward course design begins with
Choosing the readings
Getting materials from the former professor
Establishing learning goals and objectives
Selecting teaching techniques
This is a quality learning objective: “Students will be able to understand experimental psychology.”
True
False
Have you taught a course, either as an instructor or TA, over the past year? If you did not instruct students in the classroom over the past year (i.e., you had an administrative role), please select “No.”
Yes
No
What department(s) was/were the class(es) you taught in over the past year?
Which of the following pedagogical topics have you used (or have not used, but plan to) in your classrooms over the past year in a university setting? In what capacity? If you have not used the topic, please type “N/A.”
Neuroscience of learning and memory
Active learning
Inclusive teaching
Collaborative learning
Instructional technology
Formative assessment
Summative assessment
Backward design
In a typical class session, what is the average percentage of class (0–20%, 21–40%, 41–60%, 61–80%, 81–100%) that you:
pause to ask for questions?
have small group discussions or problem-solving?
show demonstrations, simulations, or video clips?
Over the course of a term for one class, how many times do you have discussions on why the material is useful and/or interesting from students’ perspective?
0–2 times a quarter
Every few class sessions
At least once a week
Every session
Check all that occurred in the most recent course you taught:
Assessment of background knowledge
Students were asked to read/view material for upcoming class session
Students were asked to read/view material for upcoming class session and complete assignments or quizzes on it shortly before class or at beginning of class
Reflective activity at end of class, e.g., “one-minute paper” or similar (students briefly answering questions, reflecting on lecture and/or their learning, etc.)
Student presentations (verbal or poster)
Tried new pedagogical techniques (e.g., active learning, collaborative learning, inclusive strategies, instructional technology)
None of the above
What fraction of a typical class period do you spend lecturing (presenting content, deriving mathematical results, presenting a problem solution, etc.)?
A. 0–20%
B. 21–40%
C. 41–60%
D. 61–80%
E. 81–100%
On a scale of 1–5, please rate your level of agreement (1 - Strongly disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Strongly agree).
I feel that this class is like a close-knit family.
I trust others in this class.
I feel that students in this class care about each other.
I feel isolated in this class.
I do not feel a spirit of community.
I feel uncertain about others in this class.
On a scale of 1–5, please rate the truthfulness of these statements (1 - Completely untrue; 2 - Mostly untrue; 3 - Equally untrue and true; 4 - Mostly true; 5 - Completely true):
I could contact another student from this class if I had a question about an assignment.
If I miss a class, I know students who I could get the notes from.
I discuss events that happen outside of this class with my classmates.
I would invite people I know from this class to do things socially.
I would have discussed personal matters with students who I met in this class.
It is difficult to meet other students in this class.
No one in this class knows anything personal about me.
I rarely talk to other students in this class.
I feel comfortable seeking help from the facilitator before or after class.
I feel comfortable asking the facilitator for help if I do not understand class-related material.
I feel that the facilitator would take the time to talk to me if I needed help.
I feel that the facilitator would be sensitive to my difficulties if I shared them.
During your graduate and/or postdoc career, how much exposure have you had to an interdisciplinary community anywhere on campus? “Interdisciplinary” in this context means people from different schools on campus. For example, if you are a student in the School of Biological Sciences, how much interaction have you had with a student in the School of Education?
No exposure
Little exposure
Some exposure
A lot of exposure
Everyday exposure
On a scale of 1–5, rate the value of (1 - Not at all valuable; 2 - Slightly valuable; 3 - Somewhat valuable; 4 - Very valuable; 5 - Extremely valuable):
Learning from an interdisciplinary group
Having a supportive group with teaching as a common passion
Having a supportive group with research as a common passion
Doing activities (social, professional development, etc.) outside of your department
Opinions of people in different disciplines about your teaching
Opinions of people in your discipline about your teaching
Opinions of people in different disciplines about your research
Opinions of people in your discipline about your research
Do you have any other comments about participating in an interdisciplinary community?
On a scale of 1–5, please indicate the frequency of the following (1 - Not at all or less than one day last week; 2 - One or two days last week; 3 - Three to four days last week; 4 - Five to seven days last week; 5 - Nearly every day for two weeks):
My appetite was poor.
My sleep was restless.
I felt sad.
I lost interest in my usual activities.
I could not focus on the important things.
On a scale of 1–5, please rate your level of agreement (1 - Strongly disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Strongly agree).
In most ways my life is close to my ideal.
The conditions of my life are excellent.
I am satisfied with life.
So far I have gotten the important things I want in life.
If I could live my life over, I would change almost nothing.
Has participating in 390X affected your quality of life? If so, how?
Appendix C. Pre-PF Survey
Would you like to opt out of including your responses in our analysis?
Yes
No
Rank the value of the following aspects of your motivation for taking this class. Please select only one response per column (1 = Least important; 2 = Somewhat important; 3 = Important; 4 = Very important; 5 = Most important):
Improving quality of life
Preparing job market materials
Developing teaching skills
Having a sense of community
Building a network with PFs and the DTEI
For the following statements, click the checkbox if the statement is true.
We mostly only use 10% of our brain.
Students’ prior knowledge can help or hinder their learning.
Visual learners always learn best when they receive information visually.
Right- or left-brain dominance helps explain individual differences among learners.
Once synapses are established, they do not change.
Only the hippocampus is necessary when recalling a memory.
Learning is hindered when cognitive load is maxed.
There is evidence that effectively using active learning increases student retention of knowledge.
Using ice-breakers guarantees more participation.
Cell phones in class are ineffective active learning tools.
A student reading a passage in class is not an example of active learning.
A student’s cultural background influences learning.
Using a specific pop cultural reference is a good way to increase inclusivity.
Since course content is standardized for all students in a given course, diversity does not play a role in content delivery.
Individual students’ learning styles should govern the way material is learned.
None of the above statements are true.
Which of the following is the most likely way to engage introverted students in the classroom?
Doing student presentations
Using a reflective technique, such as a minute paper
Holding a debate
Randomly calling on students to answer questions
An educational framework that addresses individual learning differences in the classroom by encouraging instructors to implement a flexible and accessible learning environment is called:
Diversity
Collaborative learning
Universal design for learning
Open educational resource
Which of the following is an effective way of managing groups?
Only having group assessments
Having students self-select their groups
Having groups meet solely outside of class to work on their project
Implementing peer feedback as a form of assessment
When designing an activity centered on instructional technology, what is the most important thing an instructor should consider?
Student engagement with the activity
Feasibility of the activity
Alignment of the activity to the learning objectives
Time required to do the activity
Which of the following is the best example of summative assessment?
Standardized exams (MCAT, GRE, GMAT, DAT, PCAT, etc.)
Weekly quizzes
Think-Pair-Share
Pre-reading worksheets
Which of the following is the best example of formative assessment?
A cumulative final exam
A midterm exam covering one entire unit of a class
A concept check quiz given intermittently throughout a unit that isn’t worth any points
A critique paper addressing all of the readings in a class
Backward course design begins with
Choosing the readings
Getting materials from the former professor
Establishing learning goals and objectives
Selecting teaching techniques
This is a quality learning objective: “Students will be able to understand experimental psychology.”
True
False
Have you taught a course, either as an instructor or TA, over the past year? If you did not instruct students in the classroom over the past year (i.e., you had an administrative role), please select “No.”
Yes
No
What department(s) was/were the class(es) you taught in over the past year?
Which of the following pedagogical topics have you used (or have not used, but plan to) in your classrooms over the past year in a university setting? In what capacity? If you have not used the topic, please type “N/A.”
Neuroscience of learning and memory
Active learning
Inclusive teaching
Collaborative learning
Instructional technology
Formative assessment
Summative assessment
Backward design
In a typical class session, what is the average percentage of class (0–20%, 21–40%, 41–60%, 61–80%, 81–100%) that you:
pause to ask for questions?
have small group discussions or problem-solving?
show demonstrations, simulations, or video clips?
Over the course of a term for one class, how many times do you have discussions on why the material is useful and/or interesting from students’ perspective?
0–2 times a quarter
Every few class sessions
At least once a week
Every session
Check all that occurred in the most recent course you taught:
Assessment of background knowledge
Students were asked to read/view material for upcoming class session
Students were asked to read/view material for upcoming class session and complete assignments or quizzes on it shortly before class or at beginning of class
Reflective activity at end of class, e.g., “one-minute paper” or similar (students briefly answering questions, reflecting on lecture and/or their learning, etc.)
Student presentations (verbal or poster)
Tried new pedagogical techniques (e.g., active learning, collaborative learning, inclusive strategies, instructional technology)
None of the above
What fraction of a typical class period do you spend lecturing (presenting content, deriving mathematical results, presenting a problem solution, etc.)?
A. 0–20%
B. 21–40%
C. 41–60%
D. 61–80%
E. 81–100%
To what extent do you agree that you were able to do the following tasks (1 - Completely disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Completely agree)?
Prepare a statement of teaching philosophy
Prepare a curriculum vitae
Prepare a cover letter
Search for job advertisements to which you would apply
Be competitive on the job market
During your graduate and/or postdoc career, how much exposure have you had to an interdisciplinary community anywhere on campus? “Interdisciplinary” in this context means people from different schools on campus. For example, if you are a student in the School of Biological Sciences, how much interaction have you had with a student in the School of Education?
No exposure
Little exposure
Some exposure
A lot of exposure
Everyday exposure
On a scale of 1–5, rate the value of (1 - Not at all valuable; 2 - Slightly valuable; 3 - Somewhat valuable; 4 - Very valuable; 5 - Extremely valuable):
Learning from an interdisciplinary group
Having a supportive group with teaching as a common passion
Having a supportive group with research as a common passion
Doing activities (social, professional development, etc.) outside of your department
Opinions of people in different disciplines about your teaching
Opinions of people in your discipline about your teaching
Opinions of people in different disciplines about your research
Opinions of people in your discipline about your research
Do you have any other comments about participating in an interdisciplinary community?
On a scale of 1–5, please indicate the frequency of the following (1 - Not at all or less than one day last week; 2 - One or two days last week; 3 - Three to four days last week; 4 - Five to seven days last week; 5 - Nearly every day for two weeks):
My appetite was poor.
My sleep was restless.
I felt sad.
I lost interest in my usual activities.
I could not focus on the important things.
On a scale of 1–5, please rate your level of agreement (1 - Strongly disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Strongly agree).
In most ways my life is close to my ideal.
The conditions of my life are excellent.
I am satisfied with life.
So far I have gotten the important things I want in life.
If I could live my life over, I would change almost nothing.
Appendix D. Post-PF Survey
Would you like to opt out of including your responses in our analysis?
Yes
No
Rank the value of the following aspects of your motivation for taking this class. Please select only one response per column (1 = Least important; 2 = Somewhat important; 3 = Important; 4 = Very important; 5 = Most important):
Improving quality of life
Preparing job market materials
Developing teaching skills
Having a sense of community
Building a network with PFs and the DTEI
For the following statements, click the checkbox if the statement is true.
We mostly only use 10% of our brain.
Students’ prior knowledge can help or hinder their learning.
Visual learners always learn best when they receive information visually.
Right- or left-brain dominance helps explain individual differences among learners.
Once synapses are established, they do not change.
Only the hippocampus is necessary when recalling a memory.
Learning is hindered when cognitive load is maxed.
There is evidence that effectively using active learning increases student retention of knowledge.
Using ice-breakers guarantees more participation.
Cell phones in class are ineffective active learning tools.
A student reading a passage in class is not an example of active learning.
A student’s cultural background influences learning.
Using a specific pop cultural reference is a good way to increase inclusivity.
Since course content is standardized for all students in a given course, diversity does not play a role in content delivery.
Individual students’ learning styles should govern the way material is learned.
None of the above statements are true.
Which of the following is the most likely way to engage introverted students in the classroom?
Doing student presentations
Using a reflective technique, such as a minute paper
Holding a debate
Randomly calling on students to answer questions
An educational framework that addresses individual learning differences in the classroom by encouraging instructors to implement a flexible and accessible learning environment is called:
Diversity
Collaborative learning
Universal design for learning
Open educational resource
Which of the following is an effective way of managing groups?
Only having group assessments
Having students self-select their groups
Having groups meet solely outside of class to work on their project
Implementing peer feedback as a form of assessment
When designing an activity centered on instructional technology, what is the most important thing an instructor should consider?
Student engagement with the activity
Feasibility of the activity
Alignment of the activity to the learning objectives
Time required to do the activity
Which of the following is the best example of summative assessment?
Standardized exams (MCAT, GRE, GMAT, DAT, PCAT, etc.)
Weekly quizzes
Think-Pair-Share
Pre-reading worksheets
Which of the following is the best example of formative assessment?
A cumulative final exam
A midterm exam covering one entire unit of a class
A concept check quiz given intermittently throughout a unit that isn’t worth any points
A critique paper addressing all of the readings in a class
Backward course design begins with
Choosing the readings
Getting materials from the former professor
Establishing learning goals and objectives
Selecting teaching techniques
This is a quality learning objective: “Students will be able to understand experimental psychology.”
True
False
Have you taught a course, either as an instructor or TA, over the past year? If you did not instruct students in the classroom over the past year (i.e., you had an administrative role), please select “No.”
Yes
No
What department(s) was/were the class(es) you taught in over the past year?
Which of the following pedagogical topics have you used (or have not used, but plan to) in your classrooms over the past year in a university setting? In what capacity? If you have not used the topic, please type “N/A.”
Neuroscience of learning and memory
Active learning
Inclusive teaching
Collaborative learning
Instructional technology
Formative assessment
Summative assessment
Backward design
In a typical class session, what is the average percentage of class (0–20%, 21–40%, 41–60%, 61–80%, 81–100%) that you:
pause to ask for questions?
have small group discussions or problem-solving?
show demonstrations, simulations, or video clips?
Over the course of a term for one class, how many times do you have discussions on why the material is useful and/or interesting from students’ perspective?
0–2 times a quarter
Every few class sessions
At least once a week
Every session
Check all that occurred in the most recent course you taught:
Assessment of background knowledge
Students were asked to read/view material for upcoming class session
Students were asked to read/view material for upcoming class session and complete assignments or quizzes on it shortly before class or at beginning of class
Reflective activity at end of class, e.g., “one-minute paper” or similar (students briefly answering questions, reflecting on lecture and/or their learning, etc.)
Student presentations (verbal or poster)
Tried new pedagogical techniques (e.g., active learning, collaborative learning, inclusive strategies, instructional technology)
None of the above
What fraction of a typical class period do you spend lecturing (presenting content, deriving mathematical results, presenting a problem solution, etc.)?
A. 0–20%
B. 21–40%
C. 41–60%
D. 61–80%
E. 81–100%
On a scale of 1–5, how has your confidence changed regarding (1 - Completely lowered; 2 - Lowered; 3 - Did not change; 4 - Rose; 5 - Completely rose):
Developing a training program?
Creating student-centered learning outcomes (SLOs)?
Aligning activities with SLOs?
Assessing SLOs?
Facilitating discussion?
Are you likely to facilitate workshops in the future?
Yes
No
Maybe
Did you have experience developing workshops/programs before PF?
Yes
No
To what extent do you agree that you were able to do the following tasks BEFORE you started 390C (1 - Completely disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Completely agree)?
Prepare a statement of teaching philosophy
Prepare a curriculum vitae
Prepare a cover letter
Search for job advertisements to which you would apply
Be competitive on the job market
To what extent do you agree that you were able to do the following tasks AFTER you completed 390C (1 - Completely disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Completely agree)?
Prepare a statement of teaching philosophy
Prepare a curriculum vitae
Prepare a cover letter
Search for job advertisements to which you would apply
Be competitive on the job market
After serving on the PF interview committee, how confident are you (1 - Not at all confident; 2 - Somewhat confident; 3 - Confident; 4 - Very confident; 5 - Completely confident):
In your ability to understand the interview process?
In developing your own interviewing skills?
On a scale of 1–5, please rate your level of agreement (1 - Strongly disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Strongly agree).
I feel that this class is like a close-knit family.
I trust others in this class.
I feel that students in this class care about each other.
I feel isolated in this class.
I do not feel a spirit of community.
I feel uncertain about others in this class.
On a scale of 1–5, please rate the truthfulness of these statements (1 - Completely untrue; 2 - Mostly untrue; 3 - Equally untrue and true; 4 - Mostly true; 5 - Completely true):
I could contact another student from this class if I had a question about an assignment.
If I miss a class, I know students who I could get the notes from.
I discuss events that happen outside of this class with my classmates.
I would invite people I know from this class to do things socially.
I would have discussed personal matters with students who I met in this class.
It is difficult to meet other students in this class.
No one in this class knows anything personal about me.
I rarely talk to other students in this class.
I feel comfortable seeking help from the facilitator before or after class.
I feel comfortable asking the facilitator for help if I do not understand class-related material.
I feel that the facilitator would take the time to talk to me if I needed help.
I feel that the facilitator would be sensitive to my difficulties if I shared them.
During your graduate and/or postdoc career, how much exposure have you had to an interdisciplinary community anywhere on campus? “Interdisciplinary” in this context means people from different schools on campus. For example, if you are a student in the School of Biological Sciences, how much interaction have you had with a student in the School of Education?
No exposure
Little exposure
Some exposure
A lot of exposure
Everyday exposure
On a scale of 1–5, rate the value of (1 - Not at all valuable; 2 - Slightly valuable; 3 - Somewhat valuable; 4 - Very valuable; 5 - Extremely valuable):
Learning from an interdisciplinary group
Having a supportive group with teaching as a common passion
Having a supportive group with research as a common passion
Doing activities (social, professional development, etc.) outside of your department
Opinions of people in different disciplines about your teaching
Opinions of people in your discipline about your teaching
Opinions of people in different disciplines about your research
Opinions of people in your discipline about your research
Do you have any other comments about participating in an interdisciplinary community?
On a scale of 1–5, please indicate the frequency of the following (1 - Not at all or less than one day last week; 2 - One or two days last week; 3 - Three to four days last week; 4 - Five to seven days last week; 5 - Nearly every day for two weeks):
My appetite was poor.
My sleep was restless.
I felt sad.
I lost interest in my usual activities.
I could not focus on the important things.
On a scale of 1–5, please rate your level of agreement (1 - Strongly disagree; 2 - Disagree; 3 - Neither disagree nor agree; 4 - Agree; 5 - Strongly agree).
In most ways my life is close to my ideal.
The conditions of my life are excellent.
I am satisfied with life.
So far I have gotten the important things I want in life.
If I could live my life over, I would change almost nothing.
Has participating in the PF Program affected your quality of life? If so, how?
