Data-driven iterative refinements to educational development services: directly measuring the impacts of consultations on course and syllabus design

Abstract

Evidence-based practice in educational development includes leveraging data to iteratively refine center for teaching and learning (CTL) services. However, CTL data collection is often limited to counts and satisfaction surveys rather than direct measures of outcomes. To directly assess impacts of consultations on course and syllabus design, we analyzed 94 clients’ syllabi (32 faculty, 62 graduate students and postdocs) before and after consultations. Faculty and non-faculty clients demonstrated significant change following consultations (6% and 10% gains in syllabus rubric scores, representing 50% and 31% of possible gains and effect sizes of 0.73 and 1.04 standard deviations, respectively). We compared faculty clients to quasi-experimental control groups that did not receive consultations. Syllabi from non-clients scored lower and did not demonstrate similar changes across semesters. Attendance at a CTL seminar on course and syllabus design did not explain variation in clients’ syllabi. We discuss implications for assessment of CTL services and how we leveraged formative assessments to inform and iteratively refine our educational development practices.

Keywords

educational development, formative outcomes assessment, centers for teaching and learning

How to Cite

Hershock, C. & Pottmeyer, L. O. & Harrell, J. & le Blanc, S. & Rodriguez, M. & Stimson, J. & Walsh, K. P. & Weiss, E. D., (2022) “Data-driven iterative refinements to educational development services: directly measuring the impacts of consultations on course and syllabus design”, To Improve the Academy: A Journal of Educational Development 41(2): 9. doi: https://doi.org/10.3998/tia.926

39

Views

3

Downloads

Share

Authors

Chad Hershock (Carnegie Mellon University)
Laura Ochs Pottmeyer (Carnegie Mellon University)
Jessica Harrell (Carnegie Mellon University)
Sophie le Blanc (Carnegie Mellon University)
Marisella Rodriguez (University of California, Berkeley)
Jacqueline Stimson (Carnegie Mellon University)
Katharine Phelps Walsh (Carnegie Mellon University)
Emily Daniels Weiss (Carnegie Mellon University)

Download

Issue

Publication details

Dates

Licence

Creative Commons Attribution-NonCommercial-NoDerivs 4.0

Identifiers

Peer Review

This article has been peer reviewed.

File Checksums (MD5)

  • PDF: c37a554e41c1294e718c903f62b0e465