Research Questions

While we designed the library research guide applying the Universal Design for Learning Guidelines (CAST, 2018) and accessible design principles, we wanted to ensure the guide was accessible and usable for all students. The following questions guided the study design:

• Do students with disabilities perceive the library research guide to be accessible?

• Do students with disabilities perceive the library research guide to be usable?

Recruitment

The research team worked in collaboration with the Disability Resource Center (DRC) to recruit participants for this study. Eligible participants were undergraduate students with any disability. We did not limit the scope of disabilities included, as an inclusive approach would best represent the diverse needs of students. However, we recognized that the study did not represent all disabilities nor identify all barriers.

We made a recruitment flyer and placed it in locations frequented by students with disabilities, including the library’s Accessibility Studio and the DRC main office, as well as in the DRC Newsletter. Our collaborator at the DRC also sent the flyer directly to four affiliated student groups. We gave students who participated a ten-dollar gift card for their participation.

The final group of participants included six undergraduate students with disabilities. We selected every student who responded to the call for participation. Once the recruitment process was completed, we shared the names of participants with the DRC to request a list of the disabilities represented by the participants. The DRC only shared a list of disabilities for the group, to ensure privacy and anonymity. Participants’ names were not linked to specific disabilities, the interview responses, or to any of the audio/screen capture recordings.

Participants

Six undergraduate students with disabilities participated in the study. The participants represented a range of characteristics that include sensory, physical, and cognitive disabilities. Each participant disclosed at least one of the following disabilities to the DRC: attention deficit hyperactivity disorder, autism spectrum disorder, cerebral palsy, generalized anxiety disorder, major depressive disorder, and visual impairment. Although some disabilities were apparent during the study, we did not associate individual disabilities with participants or their data.

Although the number of participants represented a small percentage of the 5,500 students registered with the DRC, the six participants were close to usability testing standards according to Nielsen (2012), which recommends five users to gain insights in a usability study. We recognized that our study alone could not represent the diverse disabilities of all students or address all accessibility issues.

Measures and Materials

Our accessible UX study included three task scenarios and an interview with ten questions (see appendix). The DRC representative reviewed the test script, including tasks and questions, to better ensure our neurodiverse participants would clearly understand the directions. We recorded the tasks and interviews using screen and audio recording tools (Morae software, Yeti USB microphone) for qualitative analysis.

Procedure

In preparation for the study, we asked participants to indicate if they would need additional accommodations. Assistive technologies, such as MAGic (computer magnification program), JAWS (screen reader program), and Kurzweil (text to speech software), were available to participants through university-wide subscriptions. We also provided an adjustable height desk, a large key keyboard, noise-canceling headphones, and a trackball mouse that were available from existing library equipment. In addition, we offered a printed version of the tasks and questions and gave participants the option to provide written responses during the interview.

Throughout our recruitment process, we informed all participants about the purpose of our study to improve the accessibility and usability of a library research guide; that they would interact with the guide to complete tasks, using assistive technologies if needed; and that we would interview them about their experience. We repeated this information upon their arrival and at the beginning of the test to ensure participants felt comfortable with the process. All participants provided informed consent before testing began. To limit any potential test anxiety, we emphasized to our participants that this was not a test and that some barriers could prevent them from completing the tasks.

We directed the participants to think aloud while completing the scenarios, a standard method for usability testing. This method occasionally prompted both the interviewer and observer to ask follow-up questions and identify issues with the research guide. Participants had 20 minutes to complete the task scenarios, which included a flexible task of self-directed exploration that was not assigned a success/failure rating. Each participant explored the guide in their own unique way, but all participants indicated they were ready to move on to the interview at the end of the allotted time.

We limited the number of tasks based on our considerations of assistive technology use and participant energy levels. We chose the most important tasks for participants to complete first, and we included an exploratory task for the remaining time which allowed us to observe their preferred way to interact with the guide. After completing the tasks, we interviewed participants and encouraged them to clarify their points on the guide itself.

Once the UX testing was complete, the data analysis process began with transcribing the six interviews and recording successes/failures of the guide. We analyzed the interview transcripts using deductive thematic analysis (Kuckartz, 2019; Tracy, 2019). We chose the thematic categories based on the accessibility principles from WAI (2018) and usability metrics from the International Organization for Standardization (2018). See Table 1 for definitions and examples. Once we defined the categories, we reviewed and manually coded the transcripts together, frequently confirming our codes in relation to the category definitions. We combined the coded passages into a list under each thematic category to identify patterns and summarize the results.

Perceivable Information and User Interface

Participants experienced notable barriers perceiving the user interface of the guide. One participant had trouble seeing the color blue and could not see the links in the navigation menu nor within the body of the guide. The participant also noted that they often rely on a text resizing widget, and they suggested adding one to the guide. A second participant suggested increasing the font size of the headings. We made changes to the guide based on these findings, such as increasing the font size of the navigation menu, headings, and body text. Although we did not change the color of links, we strengthened their weight to increase visibility.

Operable User Interface and Navigation

Participants found the guide easy to navigate, mentioning the side navigation menu, clear and relevant headings, and the ability to find the information they needed without getting lost. One participant stated:

I thought it was pretty easy to navigate through. I thought the tabs on the side were clear enough. When you clicked on one, it would take you to a certain part of the page, so you didn’t have to dig through the whole page to find what you wanted, so that was nice.

However, one participant noticed that some of the sub-menu links jumped to the middle of the guide page where there were no headings, leading to confusion about the location of the needed information. Based on these findings, we made sure every section had a heading and that they linked correctly to the sub-menu. We also added next and previous buttons to aid users in sequentially navigating the guide.

Understandable Information and User Interface

Overall, participants easily understood how to use the guide and understood the content. One participant shared, “The guide was very easy to use and read. Everything worked just like I thought it would, and I’m sure if I was looking for some help with research, this would be a great guide to get started.” However, one participant shared that both the embedded Adobe Spark page and ThingLink learning objects were confusing to them. We used Adobe Spark and ThingLink, open interactive media software, to add dynamic content to the library research guide. See Figure 1 for an example of the ThingLink and Figure 2 for an example of the Adobe Spark page. The participant believed that if they had known how to read the Adobe Spark page, it would have helped them better understand the guide.

Figure 1 ThingLink learning object with expandable hotspots depicted by red circle with plus signs.

Figure 2 Adobe Spark page.

Although we did not make any changes to the guide, since the Adobe Spark page offered an alternative format for users, we plan to review it further and possibly integrate its content into the guide itself so that users better understand how to use it. We updated the ThingLink to feature our new discovery tool and added emphasis to the hotspots.

Robust Content and Interpretation

One participant tested the guide’s compatibility with the Kurzweil screen reader software available to students. While they found that most of the content on the guide was compatible with the screen reader software, they discovered that the embedded ThingLink learning object was not. When we created the learning object, we recognized the challenge of identifying parts of a user interface for users with disabilities. Although the screen reader could not read the text, we decided to keep this embedded object because of its integrated Microsoft immersive reader. We also highlighted its audio description feature in the guide page directions included above the ThingLink learning object.

Effectiveness in Achieving Goals

Regarding the usability of the guide, all participants completed the first task scenario, “Navigate through the guide to find the option to ‘ask a librarian’ for help,” however, one participant required guidance, and one participant found the answer on the Adobe Spark page instead of within the guide (see Figure 3). Participants struggled more with task two: “Navigate through the guide to find information on how to search for a book in the library catalog. Find how to check if a book is available in the library catalog.” Three of the six participants required assistance locating the embedded ThingLink learning object, and two other participants needed guidance using it. We believe that some of the confusion was due to the wording of task two itself and did not reflect the guide’s effectiveness. The task asked them to locate availability information for the book depicted on the embedded ThingLink learning object, and did not, as some participants misunderstood, require them to search the library catalog.

Figure 3 Getting Help page of the Library Research Test Guide.

Despite the needed assistance in the first two tasks, participants shared that the guide was well organized and effective for their learning during the interview. One participant stated, “The guide has gone beyond what I had expected. I actually received a lot of extra knowledge from this guide, aiding in my potential research and just having all of these resources available for topic explorations.”

Efficiency in Achieving Goals

Due to the focus on accessibility and the potential use of assistive technologies, we decided not to collect data about time on task. However, participants were able to review the entire guide within the time provided for exploration (task three). Participants noted that what they learned from the guide would help them find information faster. One participant stated, “Links to databases were especially good because then I could get to the scholarly research more quickly than if you just said try looking for some databases. These links are very valuable.”

Satisfaction of User Experience

The participants expressed their overall satisfaction with the guide, using statements such as “it’s a really good base” and “it is definitely awesome,” specifically mentioning the organization and content meeting their research needs. One participant stated, “I really liked how getting help was really easy… I looked at everything, and [the guide] has pretty much everything you need to write a research paper, so that’s great.”

Suggestions for Improvement

Invited to share ideas for improvement, participants believed the guide could benefit from adding instructional videos, incorporating visual design elements, modifying specific examples, and expanding content. Participants mentioned additional content would be helpful in the sections about evaluating sources, primary and secondary sources, and citation management. One participant also suggested adding an orienting video that explains the guide itself. We added content to several pages, changed examples, and linked to additional resources based on these suggestions. We plan to add fully accessible videos in the future.

Key Takeaways

The following points are suggestions for conducting your own accessible UX study:

• Design the research study, and all related materials, with accessibility in mind, building in flexibility and providing options when possible (e.g., include all study details on recruitment flyers).

• Partner with the Disability Resource Center or other accessibility experts on campus to assist with research design and recruitment (e.g., communication with student organizations).

• Communicate all details of the study and provide opportunities for participants to ask questions and share their accessibility needs throughout the study (e.g., pause for questions during interviews).

• Remove any known barriers and provide assistive technologies in the test environment (e.g., provide adjustable height desk).