1. Get Administrative Approval in Advance

Even for a small user testing program, it’s important to start planning well in advance. It took us a little over a full semester to prepare for our UX Café. Lizzy developed a detailed proposal in August 2022 and delivered a pitch to library administration at the end of that month. She proposed that a small budget, simple setup, and minimal time commitment would produce outsize results for student success. Happily, our library administrators were supportive and agreed to a modest budget. Shortly after the pitch, Lizzy submitted the user testing project for Institutional Review Board approval. They determined the project didn’t qualify as human subjects research and was solely for program improvement, so approval wasn’t needed. This may or may not be the case for other user testing programs at academic libraries, which is another reason to start planning early. We spent the rest of the fall semester completing the website migration, reviving the dormant User Interfaces Working Group, and recruiting collaborators.

2. Recruit a Cross-Library Team

The working group was re-established in January 2023 for the purpose of facilitating user testing and implementing changes based on findings. The online learning librarian, Lizzy, was chair, and the other members were three research and instruction librarians (including Hesper and Zia), three members of Library IT, the library’s graphic designer, and the new discovery and systems librarian. User testing teams absolutely do not need to be this large, but it is important to bring together stakeholders from different units.

The group purposefully included a cross-section of library faculty and staff, each with different areas of expertise and experience. Research and instruction librarians work most closely with users, Library IT has technical knowledge, the graphic designer understands web design, and the discovery and systems librarian knows how the website works in conjunction with other library systems.

Only a small subset actually conducted the user testing (Zia, Hesper, and Lizzy), with other members substituting when one of the core team was out. The other members attended meetings to identify priorities, stay informed on testing updates, provide input on decisions or roadblocks, and help turn findings into real changes. Not every team member needs to play an equal part, but it is beneficial to get input and perspectives from a variety of library employees.

3. Define Roles and Responsibilities Based on Preferences and Strengths

At the start of testing, we experimented with different roles in order to figure out the responsibilities of each one and our own preferences. We settled on recruiter, notetaker, and facilitator. After our early UX sessions, we discovered that Hesper liked to recruit and wasn’t shy about getting attention from headphone-clad students, Zia was good at capturing detailed notes, and Lizzy enjoyed talking with students while running the tests.

Our core group of three also divvied up duties to prepare for our weekly practice. Zia took the lead on finalizing the script and creating note-taking forms. Lizzy managed the buying, delivery, and reimbursement paperwork for the student incentives (i.e., snacks). Occasional absences allowed us to cover for each other and invite other colleagues in the library to join us, training them on one of the three jobs and orienting them so they could suggest new tests for future weeks.

4. Schedule in Advance

It was helpful for us to have recurring times for user testing, reserving hours on our calendars far in advance. At first, we had our UX sessions at 10 a.m. every Tuesday morning, which was helpful for remembering to prepare and meet, but meant we were approaching the same library users week after week. The second semester, we began switching off Tuesday and Wednesday mornings in order to reach a more mixed group of students. It also helped us to reserve time at the beginning of each week for a planning meeting, where we chose a focus for the week’s test, brainstormed questions, and figured out logistics for running the test.

Scheduling a longer amount of time than needed—1.5 hours when we usually finished much sooner—allowed us to stay calm and not worry if recruitment was slow on a particular day. After each tester finished, we had the leisure to briefly check in with each other to mention observations and tighten up our process. Once we reached five participants, we again checked in to see if we wanted more participants that day, depending on the time available and the task. When finished, we discussed our observations to capture some of our impressions right away and start planning how we might approach related testing the following week. We also backed up any data that same day.

Some tests needed more processing, so one or more of us would volunteer to analyze the data later in the week. For example, when we tested users on our room reservation system, we needed to time how long it took them to complete each task. We each rewatched a set of recordings and made a table of the tasks and times. Although you likely won’t need to process data every week, it’s useful to schedule time to do it soon after the test so that you can still remember what happened.

5. Be Ready to Repeat Easily

With lean UX and our small budget in mind, it was important to work with the resources we already had and make our process easy to repeat week to week. We enabled this in a few ways:

• Keep supplies together: We kept all our supplies (snacks, drinks, consent forms, pens, hand sanitizer, etc.) together in a tote bag so it would be easy to take downstairs to the library entrance.

• Make checklists: We created procedural checklists for prep, day-of, and post-test periods so we wouldn’t forget anything.

• Write scripts: We had a script for the facilitator to read when introducing UX Café and that week’s test. The first few weeks of testing were iterative; it took us 3–4 weeks to draft an introductory script that we were happy with as we tried to balance brevity with clarity.

• Use templates: Before running each test, we created a Google Form for the notetaker to use. It followed the same structure as the test, so it was easy for the notetaker to follow along and keep notes organized by question.

• Have a dedicated laptop: Our library lends laptops to students, so Lizzy borrowed one of those to maintain as a neutral laptop so students could complete tests without browser bookmarks, desktop files, or other customizations. She made sure the laptop was charged and loaded with the day’s test and that any recorded sessions were backed up afterward to Box.

• Use shelf-stable snacks as incentives: Lizzy bought nut bars, granola bars, canned cold brew and bottled iced tea in bulk. Though donuts and hot coffee might seem more appealing, having shelf-stable options was much easier on us and our budget, since we didn’t have to go shopping each week, could reuse leftovers, and could set up quickly. Luckily, students didn’t seem to mind and were often drawn toward our table by the bottles of green tea, which many liked saving for later.

6. Have a Growth Mindset and Sense of Humor

Our UX Café did not go perfectly—far from it! Challenges included adjusting to absences among our core team’, managing community members stopping at our table wanting to chat, struggling to recruit participants, finding our UX Café sign missing more than once, and hearing surprisingly loud screeches from people’s shoes walking across our echo-prone lobby on our audio recordings.

There were also times when human error tripped us up. At first we didn’t use incognito mode for tests, so searches that prior users had done popped up as suggestions. Sometimes we hadn’t done enough preparation, and we were surprised about something on our own website (e.g., we weren’t completely familiar with the process for new users to reserve rooms because the process for library staff looked different). Occasionally the facilitator forgot to start the screen recording or veered off script when something unexpected happened. At times it was hard for the notetaker to hear and record everything that happened.

Whenever challenges came up, no matter what caused them, we saw them as learning opportunities. Instead of beating ourselves up or feeling like our work wasn’t worthwhile if it wasn’t perfect, we had a good laugh and reminded ourselves that we always got something of value out of every session. What we learned may not have been what we’d expected, but these lessons helped us move forward nonetheless.

7. Keep at It

Sometimes you’ll see quick results, sometimes not. That can be frustrating! Just like we had to be flexible with the user testing itself, we also had to be flexible with effecting change. Though we were able to make a number of improvements to the library’s online presence, we are still working to improve the way we communicate our results to the library faculty and staff more generally. For example, despite rewriting the instructions for reserving a room to explicitly address known pain points, we realized months later that many librarians were still not aware of those pain points nor the new instructions. When there are many stakeholders, some stretched quite thin, it’s important to be patient. However, it’s also important to keep your obligation to provide users with the best experience possible. Be understanding of everyone’s time and obligations, but don’t let your hard work go to waste if you can help it.

If you’re interested, our materials, examples, and photos are in this Public Access folder.