Current Legal Reality

To start on a sober note, because of the lack of accessible digital materials, students have brought many court cases against higher education institutions, and some mention the libraries as playing a part in the discrimination. For example:

• The resolution agreement between the University of Montana and the U.S. Department of Education, Office for Civil Rights (2014), specifically stated that their library’s website and “search engine”—used to search the journal articles, collections, and the like—must be compliant with the WCAG version of that time.

• The settlement between Penn State University and the National Federation of the Blind (2012) had similar language.

• The resolution agreement between the University of Phoenix and the U.S. Department of Education, Office for Civil Rights (2015) said that the library resources would be subject to an accessibility audit and must be compliant with the WCAG standard at that time.

• The Consent Decree between Atlantic Cape Community College and National Federation of the Blind had similar language and added, “ACCC is not responsible for the inaccessibility of third-party databases, but shall, upon request, provide timely equally effective alternate access to the content requested” (2015, p. 7).

Past Research on Accessibility of Library Materials

The process we followed to identify WCAG 2.1 failures builds upon prior research aimed at helping librarians and library staff make their websites more accessible. Spina (2019) wrote about the need for such audits but acknowledged that achieving accessibility requires “significant investment in and prioritization of web accessibility” (para. 29), which may not always be possible due to limits on time and money. In a review of accessibility-related literature, Kimura (2018) found that people historically limited audits to technology with a “persistent lack of direct engagement with users with diverse abilities” (p. 431). Peters and Bradbard (2010) also examined the barriers to audits from an organizational perspective and concluded that money, time, and organizational priorities can put such audit efforts on hold.

Barriers to auditing do not mean that libraries want to exclude users, but rather they might lack the knowhow or process to do so. As Rosen put it, “accessibility is unique in that people often agree on its value…but may disagree on its meaning” (2017, para. 4). Ng (2017) focused on the need for libraries to incorporate accessibility into the holistic web design. Ng cited Shneiderman and Hochheiser definition of incorporating accessibility as “[implying] that diverse users…can successfully apply technology to get what they need in life” (2001, p. 367, as cited in Ng, 2017). Continuing the thread, Kimura (2018) posed the question of “what if [accessibility] barriers were never erected in the first place? Born-accessible technologies and environments are the logical outcome of a user-centered, universal design process that places accessibility at the core” (p. 432). As more libraries move toward a user-centric model of services, some sought to answer this question through user experience (UX) research.

For well over fifteen years, libraries researched making digital library materials accessible, and UX research helped them identify places for improvement by involving the perspective of users. Over the past two decades, libraries moved toward integrating user-centered design into general projects (Luca & Narayan, 2016), specific collections (e.g. digital image collections) (Comeaux, 2008), and service models (Godfrey, 2015; Logan & Everall, 2019). However, Kumar and Owston (2016) argued that only conducting accessibility evaluation with specific disabilities in mind “…may result in the perception that consideration of e-learning accessibility is necessary only in special cases and is not an essential proactive consideration for all students when e-learning environments are designed” (p. 264). So including perspective from a variety of users through UX methods can help libraries identify potential issues with their website.

Beyond library websites, courts disagree on whether libraries must ensure accessibility of vendor products. Researchers such as DeLancey (2015), Oud (2016), and Stewart et al. (2005) focused on evaluating the accessibility of vendor-supplied documentation, since libraries often use vendor products to provide core functions. They generally concluded that vendors “are not fully aware of the accessibility issues in their own products” (DeLancey, 2015, p. 109). Ng (2017) and Spina (2019) pointed out libraries have a responsibility to ensure that their websites are accessible as part of the whole user experience. In this article, I will form a bridge between Spina’s research on organizational barriers to doing audits and Ng’s research on implementing accessible changes by detailing the process we used at my library to go from needing an audit to having the documentation of what to change.

Scope of Our WCAG Audit

We audited for both WCAG A and AA level standards because common court rulings require compliance with them. However, courts do not currently require the AAA level standards because users normally can access current digital services and resources on websites fulfilling the other two levels. However, courts usually consider WCAG’s lowest level, A, insufficient by itself to meet ADA and Section 508 legal requirements. We may address the AAA level standards in the future; however, we wanted to focus our improvement efforts on standards that apply across the board.

Our website is a complex mix of web pages, PDFs, and links to resources hosted on vendor-supplied digital resources, but we only audited web pages and PDFs with our university URL stem. As with many other libraries, we use multiple vendor digital resources including our catalog and our study room reservation systems. However, we audited only digital materials that our website librarian could change or request our university Information Technology Division to change during this study. Because vendor products can be non-compliant, and court rulings disagree with whether libraries are responsible for the accessibility of vendors’ products, we will plan follow-up studies on vendor products we currently use.

Project Constraints

As Peters and Bradbard (2010) and Spina (2019) noted, barriers to doing accessibility audits usually include money, time, and organizational priorities. We faced the same barriers when starting this project, though each impacted the project in a different way.

From the start, while we had organizational approval to do this project, we had no assigned budget for the project. Because of my familiarity with website coding and WCAG standards, we decided that we could save money by not hiring a company to do the audit for us. This would preserve our unit’s budget, which we mainly use for UX research incentives. We also only used free-to-use tools, so we could avoid incurring costs to the unit’s budget. This served us well when, in the middle of the project, we moved to remote work because of the COVID-19 pandemic, and our university requested we make no new purchases because of uncertainty of what the future held.

We had no pressing outside factors determining when we needed to complete certain steps, so we set our own timeframe for completing the audit and implementing the changes. However, we did want to get it done before summer break, so the website librarian could implement changes over the slower summer semester. We proposed this project to our website librarian and other invested units in December 2019. The website librarian compiled an initial list of web pages and linked PDFs for us to audit, which she gave to our unit in the middle of January 2020, and we began our audit at the end of January. We initially slated this project for completion by the end of March 2020, but the COVID-19 pandemic and subsequent adjustment to workflows resulted in us extending the timeline by almost a month. Because I had to balance other job responsibilities, I could only devote roughly two hours per day to this project. Combined, the student assistant and I worked on this project for approximately 10–20 hours per week from late January 2020 to April 2020, meaning we spent between 120 and 240 hours to complete this audit of over 300 web pages and PDFs and to write the report.

We had one full-time staff member and one part-time student assistant assigned to this project in addition to their other duties. Leadership considered auditing our website for accessibility as falling within our scope of practice as a UX unit. Additionally, legal cases against peer institutions removed some organizational resistance. However, there were organizational barriers to training more full-time staff or faculty to assist with this, so we could not have more fully trained auditors working on this project. Nielsen (1994) recommended having three to five evaluators inspect the user interface and then aggregating their answers. However, he also said only one or two people auditing does not mean that the audit is useless, but rather that one runs the risk of not uncovering some major issues (1994, para. 2). To get things done on a timely basis, the student assistant and I audited different sections—assigned based on ability—though I spot-checked the student assistant’s work for completeness and accuracy.

Accessibility Audit Methods

As our unit had never done an audit like this before, to start we turned to Nielsen’s foundational 1994 white paper about how to conduct a heuristic evaluation. While Nielsen’s white paper was not specifically about accessibility auditing, his three specifications about what evaluators should do helped us figure out where to start:

• Explain [why something fails] with reference to the heuristics or to other usability results.

• Try to be as specific as possible.

• List each usability problem separately (1994, para. 11).

Based on research after the fact, our method matched what others had done more recently in non-library fields; for example, Alsaeedi (2020) summarized four different approaches commonly used to evaluate web pages:

1. Automated. “…runs accessibility evaluation tools on the website to gather accessibility violations against predefined guidelines” (p. 3).

2. Manual. “[Uses] human experts to examine web pages to identify violations of accessibility guidelines” (p. 4).

3. User testing. “[Identifies] accessibility issues while disabled users are interacting with the content of web pages” (p. 4).

4. Hybrid. “…combines automated and manual evaluation (human experts or with the aid of disabled people)” (p. 4).

1. Inspection methods based on evaluators inspecting a web page for its accessibility. Inspection for conformance to pre-set standards such as WCAG fall under this method.

2. Automated testing where evaluators use automated accessibility tools to check conformance of a web page.

3. Screening techniques where the evaluators use the website in such a way to simulate conditions that are typical for people with disabilities.

4. Subjective assessment where the evaluators recruit users to explore/use the website on their own and provide independent feedback.

5. User testing where the evaluators ask users to complete close- or open-ended tasks on the website while observing them (p. 249–250).

We wanted to use a process that allowed us to understand where we precisely fell short of meeting minimum WCAG 2.1 standards since those standards are used in U.S. courts of law as a marker of accessibility compliance. When examining the differences between auditing approaches, Kumar and Owston (2016) recommended conducting “student-centered accessibility evaluations” in addition to using automated tools because automated tools missed problems that arose from students’ experiences. We did consider conducting accessibility evaluations that would involve watching users interact with the website; however, we decided to delay such evaluations until we had fixed all technical issues that might prevent us from uncovering larger structural issues. Our decision to combine both tools and manual evaluation matches the same method of audit Hong et al. (2008) used when they compared web accessibility of US and Korean government websites. Unlike them, though, we did not intend to compare the severity of issues on our website to other websites; instead, we wanted to simply identify their existence to then fix them.

Terminology

In the following sections, I use some coding jargon, so here’s brief glossary:

• Elements: shorthand for any piece of or group of information on a web page plus its associated markup.

• Structural markup, or just “markup”: web-based computer code that tells a computer how to display information on a web page. Throughout this article, this term loosely refers to all types of code, including HTML, CSS, SVG, and JavaScript.

• Source code: the version of the code that end users of websites only see the results of, for example: seeing a border around an image but not the markup that tells a browser which kind of border to display.

Tools

Our website has many web pages, so at certain times we needed tools to help us audit things in bulk or audit for WCAG sections that require line-by-line analysis of code. Also, neither of our auditors were Blind nor color blind, so we used additional tools to assist us with seeing our web pages in a way somewhat similar to our users. When selecting tools, we chose to use only open-source free-to-use tools because this project had no allocated budget. This may be a familiar situation for many readers, so hopefully this brief description of tools we used during our audit provides a starting point.

We used the “inspect” feature found in most browsers for all WCAG sections that required evaluating formatting and markup code—of which there are too many to list here. This feature allowed us to see the underlying source code. Using this tool required that we understood all HTML and some CSS markup, since we used it to check markup usage against standard usage.

We also used the WAVE Web Accessibility Evaluation tool (https://wave.webaim.org) to see what web pages looked like without formatting to determine read order and whether certain content would be available for screen readers to access without the additional formatting. While this tool explains itself through help text, I recommend understanding basic HTML markup and the different ways one can code images on a website before using it.

We used our work computers’ grayscale color filter combined with WAVE to identify whether elements on our web pages had enough contrast—relevant to WCAG sections 1.4.3, 1.4.6, and 1.4.11. Grayscale color filters allowed the auditors to visually “see” web pages in a way that somewhat mimics the experience of users who cannot see color; if an element “disappeared” in grayscale, then that element had insufficient color contrast and would need to be visually redesigned for usability.

Finally, we used the World Wide Web Consortium’s Nu HTML checker (https://validator.w3.org/nu/) to validate current HTML code against accepted use—the scope of section 4.1.1 of WCAG 2.1. This section of WCAG 2.1 is very detailed, and it is outside of the scope of this article to provide a more comprehensive explanation about it or the Nu HTML checker. However, because of the tool’s comprehensiveness, we decided to use it to reevaluate the website after we had fixed the issues identified during the first audit. Libraries with personnel who understand web code might consider using this tool more extensively to identify problems on a page-by-page basis.

As a side note, we did not find a URL-based validator for svg markup. While we use svg markup on a few of our web pages, we did not have the appropriate form of svg to use existing non-URL-based validators. However, we learned that most of the material generated in svg markup—building maps—did not comply with WCAG 2.1 standards, so we evaluated whether we needed that markup or if we could use a more accessible format for that content.

Preparing

Before we began auditing, we needed to make sure that our Excel list of web pages and linked PDFs included all web pages and PDFs, and we needed to sort them. To do this, we:

1. Defined our scope of what web pages and PDFs to include: would we be auditing for only one department of our library, or would we be auditing the whole website? We decided to audit all web pages with our university’s URL stem, which means that we omitted all web pages hosted on vendor products, such as Springshare’s LibGuides.

2. Checked systematically to see if the Excel list included all the web pages. We did this by starting on our homepage main navigation and clicking on every link, and then clicking on every link on the subsequent web pages. This took several hours, but we uncovered over 200+ web pages and PDFs not included on the initial “analytics generated” list from our web librarian.

3. Categorized the web pages in Excel, which helped later when evaluating WCAG sections that only apply to specific web elements instead of entire web pages—such as forms and images. We added columns in our Excel spreadsheet for the following:

   1. Contains videos

   2. Contains forms

   3. Contains images

   4. Contains table(s)

   5. Contains elements with unknown accessibility

Training

As we began auditing the web pages and PDFs for compliance with the WCAG sections, I trained the student assistant on basics of HTML markup and on how to read and navigate the WCAG website. I trained the student assistant section-by-section, which allowed us to focus on specific elements one at a time. For example, I asked the student assistant what they knew about structural markup—section 1.3.1 Info and Relationships; they had little prior knowledge, so we began working through each technique and failure. I referred heavily to w3schools (https://www.w3schools.com) during my explanations, and I would recommend it as a resource for those needing to teach themselves. Because I understood WCAG, having worked with version 2.0 previously, and the student assistant had never worked with it, I coached them through auditing the first one or two web pages per standard. Later, I spot-checked their work after they finished auditing a section.

The WCAG website’s short “how to meet [this section]” gives a good overview of each section. I encouraged the student assistant to take note of the “techniques” prefixes within each section, since the WCAG website categorizes pass or fail techniques based on markup and file type. Also, their in-depth web pages for each technique give good examples of what they are and are not talking about, which made it easier for us to audit standards we had not worked with before.

Auditing

Once we completed some initial training, we began auditing. To make auditing simpler, we broke it into four phases:

1. Audit for simple WCAG sections: These sections generally applied to very specific elements on our web pages—such as videos or images—and required no prior knowledge of code. For us, this included all of section 1.2 Time-based Media and many image-based sections, such as 1.4.1 Use of Color and 1.4.7 Low or No Background Audio.

2. Audit for highly repetitive sections: Because our university uses a system-wide template and backend content management system, some issues appeared on every page. So we reduced the number of web pages that we needed to check when auditing sections that applied to all pages. These included sections such as

   • 1.4.4 Resize text

   • 1.4.10 Reflow

   • 2.4.1 Bypass Blocks

   • 3.1.1 Language of Page

3. Audit for tedious sections: These sections required some knowledge of web code and could be different on every web page, so we had to systematically check more than 300 web pages and PDFs for these sections. Generally, the WCAG techniques pages had sufficient examples for us to understand them without having to refer to other websites. For us, many sections fell under this phase including 1.3.3 Sensory Characteristics and 2.1.1 Keyboard.

4. Audit for difficult sections: Technical reasons made these sections more difficult to audit for. This included section 4.1.1 Parsing, which we had to postpone because the tool we used flagged a lot of code for review that we had previously identified. So we found it unnecessarily redundant to audit for this section until other changes were made.

We used these phases as guidelines and had no specific end date because we audited while working on our other job responsibilities. However, audit phases 1 and 2 combined took about half the time to audit as phases 3 and 4 combined. Phase 3 was especially slow to audit because it involved highly repetitive motions of copying and pasting web URLs, right clicking in the browser window to bring up the inspect element, and then scrolling through the code to evaluate it—for each web page and PDF. After a while, we felt pain in our wrists and shoulders because of the repetitive motions. For the sake of well-being, I highly recommend building in periodic breaks during that phase to alleviate carpal-tunnel pain.

Reporting

During the audit, we wrote up our findings in a Word document organized by WCAG 2.1 section—see the appendix for examples that relate to our UX web pages. This allowed us to not only keep track of what we had done but also to keep things organized for our web librarian. Report lengths will vary depending the number of web pages and problems; our report ran to about thirty-five single-spaced pages including front material discussing the need for such an audit.

In each section of the report, we summarized the problems that applied to many or all pages, and then we reported problems that uniquely impacted a specific page in a specific way. Because the noncompliance problems could be summarized by WCAG standard, and we had over 300 web pages, we decided to keep the standard-by-standard organization for the final report. For each standard, we organized problems by three categories: Global, Semi-Global, and Page Specific. We did not have many global issues because those issues usually applied to the whole website. We had many semi-global issues that affected sets of web pages, typically those authored at the same time by one person, so issues transferred from one page to the next. Finally, we also had many page-specific issues, as employees often alter web pages and add new one-off web pages.

We do not claim to have identified every non-compliant issue on our website, but patterns emerged from the hundreds of unique compliance issues. In our audit report, we summarized these major patterns with the goal that our web librarian would use them to create better guidelines for our overall website. We hope that good guidelines will make it easier to create accessible web pages from the start and make it easier to communicate the importance of accessibility to skeptical colleagues.