Introduction

As a pedagogical design framework, Universal Design for Learning (UDL) provides an inclusive pathway for student-centered learning, accounting for diverse experiences and learner variability. In library settings, UDL impacts everything from face-to-face instruction, learning online—both synchronous and asynchronous, online library collections and databases, and even unmoderated online course guides.

In Summer 2019, we made plans to update our Libraries’ most-viewed online course guide, which librarians created using Springshare’s LibGuides software, a widely used content curation platform for libraries. Video tutorials within this specific LibGuide featured outdated interfaces, and the guide’s content had remained static for years. Prior exposure to UDL led us to model our approach after librarians at Eastern Carolina University who had implemented some features of UDL in a LibGuide (Webb & Hoover, 2015). Our work builds on their foundation by extending to other UDL principles, as well as on the work of the Center for Applied Special Technology (CAST) in inclusion of UDL checkpoint-inspired elements within our guide.

By including UDL elements, we sought to benefit all learners, especially those with structural barriers. We investigated, through a usability test with seventeen students, how our UDL elements affected their ability to complete a set of ten representative tasks. We also inquired about past courses where UDL could have enhanced their learning experiences. Finally, we implemented a set of changes to our guide and determined recommendations for others seeking to incorporate more UDL principles into their own online guides.

Literature Review

Universal Design for Learning (UDL)

With roots in architecture (Nall, 2015), Universal Design has also been applied to web accessibility over the past few decades (World Wide Web Consortium, 1997), but UDL goes beyond simply making content accessible. UDL draws from neuroscience and education research, leveraging “the flexibility of digital technology to design learning environments that from the outset offered options for diverse learner needs” (Meyer et al., 2014, p. 3). Meyer and colleagues at CAST helped coin the term Universal Design for Learning, while the National Center on Accessing the General Curriculum helped develop the principles of UDL after the 1997 reauthorization of the Individuals with Disabilities Education Act (Edyburn, 2010; Rogers-Shaw et al. 2018).

From its roots in special education technology, UDL became the framework for enhancing curriculum in a way that benefits all students, amending the disconnect between a diverse student population and a one-size-fits-all education curriculum (Edyburn, 2010). The conversation shifted from talking about disabled students to disabled learning environments and the importance of creating student-centered learning experiences (Rogers-Shaw et al., 2018; Marcyk, 2015).

In his 2010 article on propositions for new directions of the UDL movement, Edyburn warned that in order to keep the momentum and make sure that UDL stayed true to its original purpose, changes needed to happen. Although many UDL-centered studies in the past decade have addressed his concerns, we particularly kept in mind Edyburn’s warnings that “UDL is fundamentally about proactively valuing diversity” (p. 36), “technology is essential for implementing UDL” (p. 38), and “UDL must be evaluated on the basis of enhanced student performance” (p. 39).

UDL and Web Accessibility

UDL’s founders advised providing multiple means toward achieving three principles: (1) representation, which ensures that information is presented to students in a variety of formats; (2) action and expression, which assures students have different means of interacting with information and numerous outlets for building new skills; and (3) engagement, which increases student active participation and motivation, while keeping a student-centered learning approach at the heart of curriculum (Kieran & Anderson, 2019; Hanesworth et al., 2019). Each principle has three guidelines, with two to five checkpoints for meeting each goal. Readers can find the entire framework in a graphic organizer from CAST (2018). The guidelines are not meant to be a checklist, but rather a set of suggestions to mix and match to meet specific learning goals and develop “expert learners” with the means to succeed in any scenario (CAST, 2018).

The World Wide Web Consortium develops standards for working across the web, including the Web Content Accessibility Guidelines (WCAG). In 2018, WCAG version 2.1 launched. Throughout their recommendations, the consortium repeats that guidelines do not cover each and every type or combination of disabilities. The first five UDL guidelines parallel the four principles of WCAG 2.1: Perceivable, Operable, Understandable, and Robust (World Wide Web Consortium, 2016). See Table 1.

Table 1:

Overlap between WCAG 2.1 principles and UDL guidelines.

WCAG 2.1 Principle UDL Guideline(s)
Perceivable Provide options for perception (1)
Operable

Provide options for physical action (4)

Provide options for expression & communication (5)

Understandable

Provide options for language & symbols (2)

Provide options for comprehension (3)

Robust Provide options for physical action (4)

WCAG and UDL guidelines do not overlap completely as their goals are different, but WCAG 2.1 guidelines offer a foundational first step when hosting learning materials or curriculum online. WCAG 2.1 provided additional success criteria specifically for cognitive, language, and learning disabilities but faced “significant challenges” in doing so (Kirkpatrick et al., 2018, as quoted in Spina, 2019). UDL’s principles of engagement and expression require responses from learners beyond the mere interaction WCAG 2.1 principles cover. Though our investigation did not focus on learning and cognitive disabilities, we did have some participants with indeterminate documented disabilities. Members of the World Wide Web Consortium have added supplemental information about these specific disabilities in a working draft titled “Making Content Usable for People with Cognitive and Learning Disabilities” (Seeman et al., 2020). We believe this work contains significant overlap with UDL, but as it is ongoing, we chose to focus specifically on UDL.

UDL in Universities and Libraries

In their book about UDL in higher education, Tobin and Behling (2018) discussed how UDL went from a widely adopted practice in K–12 in the 1990s to a higher education movement, gaining traction in the early 2000s after the US Department of Education issued grants to encourage the inclusion of UDL in classrooms. Although the promotion of UDL started in many college and university disability services offices, the grants became hugely successful and increased awareness about inclusive design and diverse learners across campuses, while improving overall student success and motivating classroom instructors to enhance their own pedagogical approaches. When instructors apply it in physical and online environments, UDL offers different formats for content delivery of information, curriculum, and instruction practices, and it supports student participation and engagement. Rogers-Shaw et al. (2018) found that it had a direct impact on the way higher education regarded learning, stating “UDL represents an epistemological shift away from individualistic approaches to the teaching-learning transaction, allowing course design and educational practice to directly address issues of justice and inclusion” (p. 28).

UDL applies to libraries in many different ways: easing navigation and access within library buildings, providing multiple ways to use reference services, considering accessibility and student choice in the classroom, and improving the library’s web presence by avoiding jargon, displaying important words prominently, and including transcripts with videos (Nall, 2015). Some of the most effective UDL strategies for librarians have included providing multiple ways to represent content (video, screenshots, written directions, etc.), making sure web content meets accessibility standards, and designing for a diverse set of student abilities (Marcyk, 2015). Recent articles described UDL’s usefulness in addressing issues of accommodation and retrofitting in library instruction (Whitver, 2020) and also focused on web accessibility for access to library materials (Peacock & Vecchione, 2020).

Webb and Hoover’s 2015 study on implementing UDL elements in library tutorials focused on UDL’s first principle, representation. While they included options for students to interact with content based on their preferred learning styles, we wanted to incorporate additional principles of UDL. We chose to not only enhance the accessibility of our guide and give students options for processing information, but also to add elements to provide more opportunities for engagement and expression. We also sought to take the focus off the contested idea of “learning styles,” instead taking cues from Meyer et al. (2014). They highlighted learner variability as a central concern of UDL, defined as the “dynamic and ever-changing mix of strengths and challenges that makes up each learner,” going so far as to say, “[p]utting learners into categories is a flawed approach both because it grossly oversimplifies and distorts the reality of those learners’ experience…and because it implies that learners in one category are somehow different from those in another category” (p. 49).

Shotick (2016) offered several ways to feature UDL guidelines by integrating UDL elements into LibGuides. To incorporate multiple means of representation, she suggested using text alternatives to videos, making PDFs accessible, using alt text for images, and approaching each page as a single lesson to create logical breakpoints. Librarians can address multiple means of expression by including quizzes to test comprehension or using the LibGuide in a class. They can incorporate multiple means of engagement by using open-ended questions to reinforce content and allowing students to contextualize the information. Based on the CAST guidelines, UDL principles, and our knowledge of how students use this specific LibGuide, we implemented some of these enhancements to address student choice and accessibility.

UDL for Social Justice and Equity

Many factors influence students’ learning experiences, and research shows a direct correlation between students’ individual background and learning. Chita-Tegmark et al. (2012) explored the impact cultural experiences have on student cognition as they shape and reshape their brains, in turn influencing all aspects of learning. Learners within particular cultures may have different learning needs while processing the same material and information.

More recently, authors have explored how multiple learning profile factors affect learning and engagement, particularly for students from historically oppressed or marginalized groups. These students often face stigma due to erroneous assumptions about their abilities and capabilities. Kieran and Anderson (2019) asserted that not only a student’s culture, but also their race, language, background knowledge, socioeconomic status, ability, and previous learning experiences affect learning. They argued one should not view these barriers to learning as a deficit on the student’s part, but rather as a shortfall of the education system. Hanesworth et al. (2019) further expanded on culture’s impact on learning, focusing on incorporating UDL and a culturally sustaining pedagogical approach to assessment within a higher-education setting. The authors argued the structures and procedures of assessment require learners to subscribe to specific cultural norms, such as knowledge and values. Students who are unfamiliar with or who do not adhere to these cultural norms may feel devalued or belittled.

To fully meet students’ learning needs, the intersectionality of culture and learning must be at the forefront of all curriculum design, and UDL provides a framework that offers multiple means to contend with learner variability. Incorporating UDL into all curriculum facets allows for differentiating instruction and assessment, thus meeting the various needs of individual learners. Including UDL elements alongside culturally responsive pedagogy reduces barriers for academically diverse learners while increasing all students’ proficiency (Chita et al., 2012). Kieran and Anderson (2019) reminded us that incorporating UDL without considering cultural perspectives and differences may “increase the disparity in student achievement” (p. 1202). Hanesworth et al. (2019) discussed critical approaches to assessment that incorporate the three main UDL principles to ensure students are assessment-literate: variations in feedback and frequent assessments (representation); student evaluations and different means of expressing what they have learned in a variety of outlets, including technology (action and expression); and timely feedback outlets that include self- and peer-assessments across various levels (engagement).

Structural barriers arise when learning environments do not accommodate cultural learning differences, but incorporating UDL into all facets of learning helps overcome these barriers. This connection formed the basis for our decision to reach out to students with different structural barriers to learning.

Background

Washington State University (WSU) is a medium-sized public research institution in the northwest United States. In fall 2012, the new University Common Requirements system launched with Roots of Contemporary Issues (History 105/305, or RCI) as the foundational, mandated undergraduate course. The University Common Requirements consist of required class categories designed to build student skill proficiency in seven learning goals (Washington State University, n.d.). RCI addresses five of these goals: critical and creative thinking; diversity; communication; depth, breadth, and integration of learning; and information literacy. Approximately 4000–5000 students enroll in RCI each year.

A significant portion of the RCI course grade is a term-length research paper, consisting of four Library Research Assignments (LRAs) designed to familiarize students with research fundamentals. Completing LRAs spaced evenly throughout the first three-quarters of the semester, students progress from general topic ideas to formulating research questions and thesis statements. Throughout the LRAs, students find sources of particular formats, describe how those sources inform their research, and cite all materials in Chicago style. Final essays are systematic accounts of historical roots of a contemporary issue across time and geographic regions and are six to ten pages in length.

The creation and delivery of the LRAs and final essay is a joint venture of the RCI Program and the WSU Libraries’ Undergraduate Services Team, which includes an RCI librarian point person. The team supports RCI students through various reference services, such as one-on-one research consultations, workshops, online chat, and email. In addition, librarians constructed RCI online research guides in concert with the LRA templates for the course’s launch in fall 2012. These online modules exist as LibGuides with one page for each of the four LRAs and a small set of special topic pages such as Chicago style citation examples and a sample annotated bibliography. The guide presents lessons either in text or video (Johnson, n.d.). Each of the LRAs have links directing students to appropriate segments of the RCI LibGuide.

In the past five years, WSU has taken significant steps to increase enrollment of first-generation college students or learners from traditionally underserved and marginalized populations. To understand the unique perspectives and backgrounds these students bring to the academic enterprise, we aimed to remove barriers from the learning experience and empower students by enhancing their research skills through the incorporation of all three UDL principles into our online instructional materials. We focus here on testing the overall effectiveness of the RCI LibGuide, especially as it pertains to the addition of UDL elements for the 2019–2020 academic year.

Framing Our Initial Research Questions

The WSU undergraduate population continues to diversify, so learning tools need to evolve to meet emerging needs. All students are unique and face barriers to learning, but some individuals experience more significant curricular and environmental challenges, the very issues UDL helps mitigate. The structural inequities we considered were accessibility needs, distance learning, and status as first-generation college students. In addition, we sought to include international and transfer students, students over the traditional age for college, students in financial need, and students from racial/ethnic minority communities that have been historically underrepresented in higher education.

We sought to answer two questions:

  1. How effectively do students accomplish common course tasks using online materials with UDL elements, especially those who face structural inequities in education?

  2. What can previous student learning experiences tell us about improving our implementation of UDL elements?

We used usability testing to assess the impact of UDL elements on completing course tasks. We asked a few short questions about previous learning experiences, and we asked about multiple means of both representation and engagement to further investigate how student preferences could shape our implementation.

Methods

Outline of Study

Our study followed these steps:

  1. Identify an appropriate course guide.

  2. Implement UDL elements, being sure to not overwhelm the user.

  3. Test implementation with representative tasks, being sure to highlight UDL elements.

  4. Ask follow-up questions about learning experiences.

  5. Assess themes and make improvements to guide.

Implementing UDL Elements

In Spring 2019, several tutorials in the RCI LibGuide needed updates due to database interface changes, so we decided to completely overhaul the guide, focusing on UDL principles. We first identified appropriate UDL elements to incorporate into the LibGuide.

We looked for examples of UDL in LibGuides or other more static tutorials and webpages, eventually finding two through CAST’s work. The first example we found was an interactive online version of Meyer et al.’s 2014 book Universal Design for Learning: Theory and Practice, which demonstrates all UDL principles in action, supplementing ninety-eight pages of text with captioned videos, tools for highlighting, and spaces for notes and learning more.

Second, we believed we could implement many UDL elements based on CAST’s version of the National Educational Technology Plan (CAST, 2010). This example showed us many ways we could include UDL elements spanning each principle with minor intervention (see Table 2). CAST notes which guidelines each element demonstrated with small “UDL” pop-ups. Implementing UDL elements for their own sake will not always benefit the end user depending on the learning outcome or curricular structure; for this reason, we aimed to change the structure as little as necessary to avoid cluttering the guide.

Our ideal goal was to implement UDL guidelines across each principle: multiple means of representation, multiple means of engagement, and multiple means of action and expression. We did not see the latter two principles in previous practical implementations of UDL elements in online learning materials in libraries.

We ultimately added six elements related to UDL Checkpoints (Table 2 and Figure 1). The first three elements focus on the principle of providing multiple means of representation. We borrowed elements 3–6 and their associated checkpoints from the CAST (2010) example, after assessing which we could implement given the constraints of both LibGuides and the RCI structure. We borrowed code from Smith (2020). We incorporated two elements focused on the principle of providing multiple means of engagement (elements 4 and 6) and one element focused on the principle of providing multiple means of action and expression (element 5).

Table 2:

UDL elements with checkpoints and implementation.

UDL Element UDL Checkpoint Implementation (see Figure 1 for examples)
1. Captions 1.2 “Offer alternatives for auditory information” Added captions to nine YouTube videos.
2. Transcripts 1.2 “Offer alternatives for auditory information” Added written transcripts of each tutorial with links to YouTube video.
3. Glossary 2.1 “Clarify vocabulary and symbols” Added page to the LibGuide with definitions for eighteen words.
4. “Learn More About” Sections 8.2 “Vary demands and resources to optimize challenge” Added two collapsing panel accordions containing supplemental topic information.
5. “Librarian Coach” 6.2 “Support planning and strategy development” Added three popover images, one within a collapsing panel.
6. “Check Your Understanding” Self-Quiz Questions 9.3 “Develop self-assessment and reflection” Added four quizzes using Springshare’s LibWizard.

Captions and transcripts for video tutorials aided web accessibility, with the goal of enhancing student learning. A glossary provided students with definitions of commonly misunderstood words and phrases. “Learn More About” sections functioned as expandable areas with supplementary topic information, and “Check Your Understanding” questions allowed students an opportunity to self-assess and receive feedback. Finally, we introduced a “Librarian Coach” as an image that users could select to receive search tips and encouragement, so users would feel a connection to the RCI Librarian. Once we had all the UDL elements in place, the next step was to monitor use of the UDL elements with LibGuides’ analytics. We also created a usability test for the guide and our target user pool.

Figure 1:
Figure 1:
Figure 1:

Screenshots of UDL elements.

  1. Captions on an embedded YouTube database tutorial;

  2. Transcript linked below each tutorial;

  3. Glossary showing entries for annotation and anthology;

  4. “Learn More” section, shown here expanded;

  5. “Librarian Coach” popover displayed after click;

  6. “Check Your Understanding” self-quiz question shown at two stages: (a) before clicking and (b) before answering.

Recruitment

After securing Institutional Review Board approval, we identified specific partner programs at WSU that engage students who have experienced the aforementioned structural barriers to education. These partners distributed our recruiting message to the students within these programs, and we sent it to all instructors currently teaching an RCI session. A total of thirty-four students replied with interest, and we screened out any who had both already taken History 105/RCI and had received library instruction during the course.

Usability Tasks and Qualitative Questions

We conducted a total of seventeen individual usability sessions: sixteen in person and one remotely using Zoom. The sessions, which occurred in January and February 2020, averaged one hour in length and began with ten tasks using the RCI LibGuide. The tasks focused on areas where students have displayed difficulties with the RCI research project, as the RCI librarian observed over hundreds of research workshops since 2012, and which would feature potential use of the new UDL elements. Two library student employees gave feedback on the tasks’ difficulty before testing began (see the list of tasks in the appendix). We asked participants to think aloud as they navigated the guide. We recorded the participants’ voices and screens; one of us read the prompts aloud and another transcribed the users’ movements and speech.

After participants finished the tasks, we asked them a short set of qualitative questions covering the effectiveness of the UDL elements and the students’ prior academic learning difficulties due to curriculum design and delivery elements (see questions in the appendix).

Results

Usability Testing: Task-Oriented Questions

Participants completed the ten tasks at varying success rates, with one task having a 100 percent success rate while another just 24 percent. The overall mean for all tasks was 73 percent (122/168) correct. We judged seven of the 168 (4 percent) task attempts as not correct or incorrect, but as a neutral “maybe.” In these cases, a participant successfully completed the task, but did not use the RCI LibGuide to find the answer, instead opting for an “alternate path.” For summative data, individual task completion success, and the exact wording for each task, refer to Table 3.

Table 3:

Usability tasks participant results.

Tasks # Success # Failure # Alternate path % Success
1. How do you email your RCI Librarian with a question about your research? [“Librarian Coach” UDL element] 10 5 2 59%
2. Weinstein, “Plato’s Republic,” 452–53. Is this an example of Chicago style bibliographic citation, full footnote, or abbreviated footnote? 15 1 0 94%
3. Your instructor/TA emails you saying that you can use a chapter from a history anthology or edited volume as part of the research for RCI.? What is an anthology? [Glossary UDL element] 13 3 0 81%
4. Your instructor/TA suggested you use The Economist as a way to find a primary source for your topic. How can you find access to that database? 12 3 2 71%
5. What are two tips for successfully using JSTOR to find history journal articles? [Transcripts UDL element] 17 0 0 100%
6. The answer to your research question becomes what important part of your final paper? [Check Understanding UDL element] 8 9 0 47%
7. What two-word label will a non-WSU book have in Search It? 4 11 2 24%
8. Your instructor/TA advised you to consult the RCI LibGuide’s annotated bibliography sample to aid you in your work. What fruit is the bibliography about? 16 1 0 94%
9. You want to learn more about any historical database so you watch its YouTube tutorial on the RCI LibGuide (please find it). How do you activate the closed captioning feature? [Captioning UDL element] 14 2 1 82%
10. How are specialized encyclopedias different from general encyclopedias? [Learn More About UDL element] 13 4 0 76%
All questions 122 39 7 73%

Most participants successfully found the RCI librarian’s email link (task one, 59 percent). Most of the unsuccessful attempts involved subjects clicking on a general help button from the RCI LibGuide homepage. The contact information box did not clearly label the RCI librarian as such, so students had to deduce the correct response.

All but one of the seventeen participants (94 percent) successfully identified a Chicago style citation footnoting element in task two. A clear Chicago style link from the main left side navigation bar of the LibGuide led most to the correct answer.

Task three asked students what an “anthology” is; 76 percent (13/17) of the participants succeeded in using the guide to find the answer. Nine participants used the glossary page on the guide, while four used the LibGuide search function.

Task four centered on finding a specific online database (The Economist) in order to discover topic-relevant primary sources; 71 percent succeeded. If participants thought to use the overarching structure of the LibGuide—which is divided by information types—books, journals, primary sources—the user progressed to the answer. If they concentrated on finding a database list or page which contained The Economist, they struggled and were not successful. Some also had difficulties when they clicked on “durable linking” from the main navigation bar, thinking that the “durable linking” page might have database links.

The fifth task asked the subjects to identify tips for using JSTOR to find scholarly history journal articles. All seventeen participants were successful; seven found the tips in the YouTube video, six scanned the video transcripts, and four read the “Specifics for Using JSTOR and Project Muse” box in the LibGuide. Multiple pathways to find the answer likely correlated to the high success rate.

The sixth and seventh tasks were the most difficult for our participants. Task six had them attempting to figure out that a “thesis statement” is the answer to a research question; only eight of seventeen (47 percent) were successful. This task could have been difficult for multiple reasons. First, its “fill-in-the-blank” nature meant keywords to search for were harder to come by. Second, participants often went to the last LRA, homing in on the task’s mention of a “final paper” at the end of the research process. The answer to this question was in both paragraph text and a “Check Your Understanding” question, but zero participants interacted with a single “Check Your Understanding” element.

Task seven asked participants to find the label for non-local books in our discovery layer: “check holdings.” Students often need to order books from other institutions, so it’s important to recognize this label. This task had the lowest success rate—only 24 percent. The reasons for the high failure likely parallel those of task six: the “fill-in-the-blank” nature of the question, the lack of a clear connection to the LibGuide navigation text (although the word “book” appears, which led some to success), and the strong temptation to leave the guide and search the discovery layer for an answer. Additionally, “holdings” is not user-centered language.

Participants did quite well on tasks eight, nine, and ten. Task eight (94 percent correct) asked participants to find the sample annotated bibliography embedded in the guide, which required them noticing the link in the main navigation bar. Fourteen participants (82 percent) successfully turned on closed captions in YouTube for task nine; many students are familiar with YouTube. Task ten asked participants to explain the difference between general and specialized encyclopedias. Most participants (76 percent) recognized where to find relevant information to complete the task because of the presence of the word “encyclopedia” in a main navigation link. The answer to task ten was in the UDL “Learn More” expandable accordion format; users were comfortable gravitating toward this type of section.

Usability Testing: Qualitative Questions

Our second research question was, “What can previous student learning experiences tell us about improving our implementation of UDL elements?” So after the users completed the ten task-oriented questions, we asked them two qualitative questions. The first focused on their immediate experience of completing the tasks: “Which UDL element helped you learn the concepts/find the answers?” The second question focused on their learning needs: “Which UDL element addresses your individual learning need?” We showed participants a screenshot highlighting our incorporated UDL elements, and we explained each element to them. See Table 4.

Table 4:

UDL element mention and use. Students could mention multiple elements.

Which UDL element helped you learn the concepts/find the answers? (Question 3a) Times mentioned (3a) Times used on relevant task Which UDL element addresses your individual learning need? (Question 3b) Times mentioned (3b)
Captions 10 15 (task 9) Captions 5
Transcripts 9 7 (task 5) Transcripts 6
Learn More 8 13 (task 10) Learn More 3
Check Understanding 8 1 (task 6*) Check Understanding 6
Coach 7 0 (task 1*) Coach 1
Glossary 6 14 (task 3) Glossary 5
Videos (not a UDL element by our definition) 3

    * Information available elsewhere on the guide.

Nine of the students highlighted both captions and transcripts as the most beneficial elements for both questions; a total of twelve students mentioned at least one of these elements as helpful for learning concepts. Transcripts addressed English language learner needs and those who described themselves as having a preference to “learn visually, rather than listening.”

Students also saw the “Learn More” element as highly relevant for finding answers, but this could have been related to task ten requiring students click on it to find the answer. Only three students said it would address their individual learning needs versus eight who said it affected their ability to find the answers.

Six students thought “Check Your Understanding” would help address their learning needs, equaling the number who thought transcripts would be beneficial.

No participants used the “Librarian Coach” element, and only one student mentioned it might address their specific learning needs, but seven students mentioned it as affecting their ability to find the answers. Our best guess is that the students like the idea of an interactive coach, but may be unaware of how it actually functions. One student incorrectly thought it started a live chat.

Students did not identify the glossary as frequently as other elements for the first question, but did note it would be helpful for their individual learning needs.

We asked participants two final questions. First, to think back to a recent time in a course where they had difficulty due to the way the information was presented and second, to recall when they wished they could have responded to or engaged with the learning material differently. We hoped to understand when students benefit most from UDL by eliciting concrete examples of recent experiences. We grouped the answers into five broad themes, ordered here by how frequently students mentioned each.

Ten students mentioned times when “auditory issues” or “assignment confusion” occurred due to how their instructors presented material—our two most prominent themes. Six students mentioned having trouble with oral-only lectures or being able to take notes while only listening and having no other reference point. Five students mentioned assignment descriptions that either lacked clear direction or examples and definitions, leading one student to say that seeing the big picture became difficult.

Three students discussed having difficulty seeing certain items in learning materials, two due to color. One student, who mentioned losing most of their eyesight over the previous year, had issues seeing anything that was presented visually.

Two students each mentioned problems related to videos and distractions. Students discussed lack of captions on videos leading to inadequate comprehension. Finally, students discussed environments with distractions, both literally—noisy physical environments—and figuratively noisy digital environments with too much information.

For our final question regarding options for engagement or response to some learning material, five students responded about needing accommodation tools including software that reads to them or a “smartpen” that records the audio of a lecture while recording movements in writing. Four students also repeated barriers aligning with presentation: the way instructors presented information impeded their understanding and made it harder to respond or engage.

A rigid class or assignment structure frustrated three students, including the strict grading of participation, confusion on what counted as an “outside source,” and wanting the option to give a PowerPoint presentation over writing a formal paper. An equal number of students discussed preferring group work over large lecture-style classes. Finally, two participants each mentioned desiring more feedback and focusing on speaking aloud rather than writing things down.

Usability Testing: Statistics and User Demographics

We anticipated using analytics to assess wider use of the UDL elements across the academic year. We tracked both caption use through YouTube’s robust internal analytics and completed “Check Your Understanding” questions through Springshare’s LibWizard. We were unable to track use of the “Librarian Coach” or clicks on the “Learn More” dropdowns, as these were built with LibGuides HTML/Rich Text asset which does not track usage.

YouTube keeps thorough statistics on average watch time, including separating by watching with captions turned on versus off. Some industry research claims captions increase watch time, including a Facebook study showing increased video ad view time of 12 percent (Facebook for Business, 2016) and a less-recent study by PLYmedia claiming average improvement of 38 percent (PLYmedia, 2009). We did not see a correlation between having captions turned on and the length of average watch time for our tutorials: the average watch time was higher when captions were turned on for only four of our nine videos.

“Check Your Understanding” questions showed low usage per pageview—around 1 percent, but together with improving how we explain the LibGuide on the guide homepage and future instruction sessions, we expect this to rise.

We found no strong correlation between our learners’ experiences of structural inequities and task completion rate, which was one of our initial research questions. As a screening question, we had asked about their experience with library instruction and the RCI class. We screened out anyone who had received library instruction specifically within the RCI class. This left participants in four categories, and those participants that had either had been exposed to RCI or had library instruction, especially in an upper-level course, seemed to have slightly higher task success rates. See Table 6.

Table 6:

Participant prior course or instruction experience and task performance.

RCI Exposure?

Received Library Instruction?

Number of Participants

Number of Successful Tasks (Avg)

None None 2 5
Currently enrolled or taken previously Yes, in English 101 6 7.3
Currently enrolled or taken previously None 5 7.8
None Yes, in an upper-level course 4 8.25

Discussion

Improving Our Guide and UDL Implementation

Based on what we learned from usability testing, we made a number of improvements to the RCI LibGuide. The usability testing participants felt the main navigation bar was the most helpful structural feature of the guide, but it was also clear we needed to update main titles and subsection titles to represent all elements of the assignment in the navigation bar. Even though the RCI LibGuide focuses on the LRAs, we realized creation of a Final Paper page was important in order to cover all parts of the research project in the LibGuide—delivering important context. Some participants wondered why there was no mention of the final paper in the LibGuide. Another navigation-oriented change was to remove “durable linking” as a main heading/page and place it under the citation page.

We made three alterations in light of discoveries about source databases. First, some students wanted a comprehensive listing of recommended databases across the LRAs, so we created a list of links to databases and our database-specific tutorials. Second, participants overwhelmingly appreciated having script text accompany audio/visual tutorials, so we decided to force captions on—viewers still have the option to turn them off. Finally, since each database has an embedded YouTube tutorial, a link to the same YouTube tutorial, a transcript of the video tutorial, and a link to the database itself, we realized we needed to label these four options more clearly. Adding new links over time without reflecting on their labels caused confusion.

We were surprised how often participants turned to the search function within LibGuides. Observing this, we noticed a few technical issues with how the search feature worked. In every result, the link with the largest and only bolded font directs to the LibGuide’s homepage, regardless of whether the searched word appears on that page. The “Key Words in Context” section of some results had the error of not actually listing the part of the page’s text where the keyword exists, or if the keyword appears multiple times on the page, results did not show the complete set of instances. When a participant searched for “economist database” the single result only showed key words in context for the word “database,” but nothing for “economist,” even though “economist” appears in multiple places across the RCI LibGuide. Finally, participants were not always able to understand what domain was being searched: the LibGuide page with the search box they used, the wider set of pages comprising the LibGuide itself, or the complete set of LibGuides at WSU. Unfortunately, we do not have local control to implement fixes concerning these issues, but we contacted Springshare about addressing them.

Matching UDL Elements to Task Success/Failure

In designing usability tasks, we attempted to strike a balance among common issues RCI students face when interacting with library resources—the main purpose of the LibGuide, use of UDL elements, and authentic tasks students complete for the class. We knew that usability testing could not ascertain that implementation of UDL led to improvement of specific learning objectives. We did want to know the flaws in our implementation. Questions about the course material and overall learning objectives aided our understanding of how to improve our guide as a whole, but they prevented us from diving as deep as we would have liked into specific UDL elements.

We experienced challenges matching UDL elements to task success/failure. We knew where students could find certain answers, resulting in successful task completion, and in six out of ten tasks, students could find the answer using UDL elements (tasks one, three, five, six, nine, and ten). Students had to use UDL elements for three of those six tasks: task three using the glossary; task nine turning on captions; task ten using the “Learn More” expandable section. Students could answer tasks one, five, and six with either UDL elements (the “Coach” popover, a YouTube video, or “Check Your Understanding” question) or other LibGuide features.

Our questions on previous course experiences elicited interesting feedback, pointing to the need for incorporating all UDL principles. Most feedback continued to focus on representation, but a significant portion of participants mentioned feedback and structure of assignments. This portion leads us to believe that students rely on motivation (engagement) and planning and monitoring progress (action and expression) to complete assignments successfully.

Challenges of Implementing UDL

While UDL asserts each learner is unique, and the framework assists all learners, our initial research focus was on those who had faced structural inequities in education. We believed we might see a correlation between certain demographics of students and successful task completion, but both the intersectionality of students’ identities and an imbalance in how many students we tested in each category, led to no definitive pattern. Structural inequities may reflect wide variability in individual students. We were glad to have students who reflected the diversity of our campus population, especially often-overlooked categories like students with disabilities (cognitive and visual) and international students.

As UDL originally arose through curriculum design, we were especially interested in updating a LibGuide positioned within our University’s core curricular goals for undergraduates. Certainly not all LibGuides are good candidates for an overhaul including each of the principles of UDL.

We also note that librarians play a supporting role to the central class, offering supplemental, library-centered guidance on how to complete the research project. When considering implementations, we specifically did not want to add more content to the already high amount of information on the page. We sought not to adjust the existing content’s presentation, opting instead to have optional text hidden by dropdown or within a “Check Your Understanding” element. Finally, when considering principles of engagement and action and expression, librarians should realize the maintenance and involvement that those implementations could create. Our passive implementations allow us to “check-in” when interested, but one could also address those principles through simple prompts in plain text.

Perhaps the biggest consideration of moving beyond simple web accessibility is making sure implementations of UDL elements remain accessible. Additional HTML or widgets, such as our “Learn More” sections or our LibWizard “Check Your Understanding” quizzes, must meet accessibility guidelines to avoid undercutting one principle by attempting to add another. We noticed in our usability tests that the “Check Your Understanding” quiz rendered dynamically as participants quickly scrolled through the content seeking answers, often loading for a few seconds longer than the rest of the page. Additionally, ensuring that both the “Librarian Coach” popover and “Learn More” sections were following WCAG 2.1 and best practices for screen reader use were top concerns.

Our usability testing and inquiries to students about previous course experience show how implementing UDL impacts learners’ abilities to complete tasks by providing multiple avenues for representation, engagement, and action and expression. Since it is difficult to draw direct lines from specific UDL elements to specific learning outcomes, we emphasize the intent of UDL to provide multiple routes toward learning goals. Future research could explore the question of learning outcomes or goals more directly through a more thorough interview process, plus pre- and post-tests. Due to our limited access to students and our investigation’s narrow focus on this specific implementation, we must leave these questions to other investigators.

Conclusion

This research project involved embedding elements reflecting UDL’s three principles into a required undergraduate course’s LibGuide. The largest number of additions align with the representation principle: captions, transcripts, and a glossary. This allowed us to optimize individual choice by providing a variety of formats for database tutorial viewing/reading. Addressing the action and expression principle, we provided support for student researcher “planning and strategy development” through the “Librarian Coach” feature. Finally, two elements reflected checkpoints for the engagement principle: the “Check Your Understanding” quiz questions provided a means for “developing self-assessment and reflection.” The “Learn More” sections also helped “vary the level of challenge” for a student’s engagement (CAST, 2018). By incorporating all three UDL principles, we believe our implementation benefits all students.

In conducting our usability testing, we assessed our design with representative users and learned from how they attempted to complete the tasks. We discovered keywords are critical to success. The participants tended to identify keywords in the task language and tried to make matches to the same words in the main navigation bar. We realized that thorough and consistent tutorial language is important to student achievement. Students also used the LibGuide search box to find answers. Task completion faltered in instances with slightly variant wording between the task language and the LibGuide text. We also noted multiple issues with the functionality of the LibGuide search box feature itself.

From our short interviews following the tasks, we drew several conclusions. First, students thought highly of the availability of video captioning and transcripts. At a time when people believe learners want to read less and view/listen more, this was quite surprising. Second, when we asked the participants about recent educational experiences where they had met difficulties, the largest number mentioned auditory issues. Students also mentioned confusion about how to properly complete assignments as a struggle, leading us to believe motivation and planning play a key role in offsetting student anxieties and barriers. While this is a difficult factor to address in the RCI LibGuide, we mitigate concerns by incorporating elements across all the UDL principles.

In terms of future directions, we believe libraries should focus on further incorporation of the engagement and action and expression principles of UDL, which often get less attention. We also aim to collaborate with our College of Education to more deeply understand UDL theory and measure impact and effectiveness of UDL, as that could benefit implementations. To aid the academic library community, we will create a LibGuide specifically focused on UDL implementation ideas. UDL offers both a theoretical underpinning and practical curricular elements which librarians can use to enhance their instructional materials and students’ learning.

Appendix

Usability Testing Tasks and Post-Testing Reflection Questions

Participant Tasks

  • Task 1: How do you email your RCI Librarian with a question about your research?

  • Task 2: Weinstein, “Plato’s Republic,” 452–53. Is this an example of Chicago style bibliographic citation, full footnote or abbreviated footnote?

  • Task 3: Your instructor/TA emails you saying that you can use a chapter from a history anthology or edited volume as part of the research for RCI.? What is an anthology?

  • Task 4: Your instructor/TA suggested you use The Economist as a way to find a primary source for your topic.? How can you find access to that database?

  • Task 5: What are two tips for successfully using JSTOR to find history journal articles?

  • Task 6: The answer to your research question becomes what important part of your final paper?

  • Task 7: What two-word label will a non-WSU book have in Search It?

  • Task 8: Your instructor/TA advised you to consult the RCI LibGuide’s annotated bibliography sample to aid you in your work.? What fruit is the bibliography about?

  • Task 9: You want to learn more about any historical database so you watch its YouTube tutorial on the RCI LibGuide (please find it). How do you activate the closed captioning feature?

  • Task 10: How are specialized encyclopedias different than general encyclopedias?

Post-Testing Questions

  1. What was challenging about using the guide?

  2. What was easy about using this guide?

  3. We added several UDL elements to enhance the guide, including captions, transcripts, “Learn More about...” sections, “Check Your Understanding” questions, a glossary, and “Librarian Coach” tips.?

    1. A principle of UDL is that there is no average learner. How did this guide’s elements (listed above) affect your ability to learn the concepts/find the answers?

    2. Thinking back to UDL components in the guide, how could they help address your individual learning needs?

    3. Thinking back to last semester, describe a time you had difficulty learning something due to the way the information was represented.?

    4. Thinking back to last semester, describe a time you wish you had options in how you engaged with, or responded to, some learning material.?

References

CAST (2010). How People Need to Learn. National Educational Technology Plan 2010 excerpt. https://web.archive.org/web/20171112140536/http:/www.cast.org:80/netp/page/NETP/l28https://web.archive.org/web/20171112140536/http:/www.cast.org:80/netp/page/NETP/l28

CAST (2018). The UDL Guidelines. http://udlguidelines.cast.orghttp://udlguidelines.cast.org

Chita-Tegmark, M., Gravel, J. W., Maria De Lourdes, B. S., Domings, Y., & Rose, D. H. (2012). Using the Universal Design for Learning framework to support culturally diverse learners. Journal of Education, 192(1) 17–22. https://doi.org/10.1177/002205741219200104https://doi.org/10.1177/002205741219200104

Edyburn, D. L. (2010). Would you recognize Universal Design for Learning if you saw it? Ten propositions for new directions for the second decade of UDL. Learning Disability Quarterly, 33(1) 33–41. https://doi.org/10.1177/073194871003300103https://doi.org/10.1177/073194871003300103

Facebook for Business. (2016, February 10). Capture attention with updated features for video ads. https://www.facebook.com/business/news/updated-features-for-video-adshttps://www.facebook.com/business/news/updated-features-for-video-ads

Hanesworth, P., Bracken, S., & Elkington, S. (2019). A typology for a social justice approach to assessment: Learning from Universal Design and Culturally Sustaining Pedagogy. Teaching in Higher Education, 24(1) 98–114. https://doi.org/10.1080/13562517.2018.1465405https://doi.org/10.1080/13562517.2018.1465405

Johnson, C. (n.d.) Roots of Contemporary Issues (HIST 105/305) Research. Washington State University Libraries. Retrieved February 1, 2020, from https://libguides.libraries.wsu.edu/rcihttps://libguides.libraries.wsu.edu/rci

Kieran, L., & Anderson, C. (2019). Connecting Universal Design for Learning with Culturally Responsive Teaching. Education and Urban Society, 51(9) 1202–1216. https://doi.org/10.1177/0013124518785012https://doi.org/10.1177/0013124518785012

Marcyk, E. R. (2015). An introduction to Universal Design for Learning in higher education for instruction librarians. LOEX Quarterly, 42(3) 8–10. https://commons.emich.edu/loexquarterly/vol42/iss3/5https://commons.emich.edu/loexquarterly/vol42/iss3/5

Meyer, A., Rose, D. H., & Gordon, D. (2014). Universal Design for Learning: Theory and practice. CAST. http://udltheorypractice.cast.org/http://udltheorypractice.cast.org/

Nall, C. (2015). Academic libraries and the principles of Universal Design for Learning: Representation beyond courses. College & Research Libraries News, 76(7) 374–375. https://crln.acrl.org/index.php/crlnews/article/view/9345/10477https://crln.acrl.org/index.php/crlnews/article/view/9345/10477

Peacock, R., & Vecchione, A. (2020). Accessibility best practices, procedures, and politics in Northwest United States academic libraries. The Journal of Academic Librarianship, 46(1). https://doi.org/10.1016/j.acalib.2019.102095https://doi.org/10.1016/j.acalib.2019.102095

PLYmedia. (2009, March 23). Subtitles increase online video viewing by 40% [Press release]. SubPLY Spoken Word Technologies. http://www.subply.com/en/news/NewsItem_SubPLY_Trial_Results.htmhttp://www.subply.com/en/news/NewsItem_SubPLY_Trial_Results.htm

Rogers-Shaw, C., Carr-Chellman, D. J., & Choi, J. (2018). Universal Design for Learning: Guidelines for accessible online instruction. Adult Learning, 29(1) 20–31. https://doi.org/10.1177/1045159517735530https://doi.org/10.1177/1045159517735530

Seeman, L., Montgomery, R., Lee, S., & Ran, R. (2020, July 17). Making content usable for people with cognitive and learning disabilities. World Wide Web Consortium. https://www.w3.org/TR/2020/WD-coga-usable-20200717/https://www.w3.org/TR/2020/WD-coga-usable-20200717/

Shotick, K. (2016). Access and Universal Design for Learning in LibGuides 2.0. In A. W. Dobbs & R. L. Sittler (Eds), Integrating LibGuides into library websites (pp. 189–200). Rowman & Littlefield Publishers.

Smith, B. (2020, July 23). Using Bootstrap in LibGuides. Cornette Library. https://infoguides.wtamu.edu/bootstrap/homehttps://infoguides.wtamu.edu/bootstrap/home

Spina, C. (2019). WCAG 2.1 and the current state of web accessibility in libraries. Weave: Journal of Library User Experience, 2(2). http://dx.doi.org/10.3998/weave.12535642.0002.202http://dx.doi.org/10.3998/weave.12535642.0002.202

Tobin, T. J., & Behling, K. T. (2018). Reach everyone, teach everyone: Universal Design for Learning in higher education. West Virginia University Press.

Washington State University. (n.d.). UCORE assessment: Student learning goals. Retrieved September 28, 2020, from https://ucore.wsu.edu/assessment/learning-goals/https://ucore.wsu.edu/assessment/learning-goals/

Webb, K. K., & Hoover, J. (2015). Universal Design for Learning (UDL) in the academic library: A methodology for mapping multiple means of representation in library tutorials. College & Research Libraries, 76(4) 537–553. https://doi.org/10.5860/crl.76.4.537https://doi.org/10.5860/crl.76.4.537

Whitver, S. M. (2020). Accessible library instruction in practice. portal: Libraries and the Academy 20(2) 381–398. https://doi.org/10.1353/pla.2020.0019https://doi.org/10.1353/pla.2020.0019

World Wide Web Consortium. (1997, October 22). World Wide Web Consortium launches international program office for web accessibility initiative [Press release]. https://www.w3.org/Press/IPO-announcehttps://www.w3.org/Press/IPO-announce

World Wide Web Consortium. (2016). Introduction to understanding WCAG 2.0. https://www.w3.org/TR/UNDERSTANDING-WCAG20/intro.htmlhttps://www.w3.org/TR/UNDERSTANDING-WCAG20/intro.html