The Budget Proposal as a Constructive Collections Engagement Tool and Practice

Background

At Minnesota State University Mankato (MNSU), the library’s Collection Management Technology (CMT) team has developed a series of reports to support journal collection analysis and other library activities. Nat Gustafson-Sundell provided an introduction and background on a new project to develop a report that can be used for journal collection review and other purposes called the Budget Proposal.¹

Gustafson-Sundell explained that the Budget Proposal was developed in response to university administrators generating new ideas about how to conduct university business. At the start of Academic Year 2024, the administration announced the university would move toward a zero-based budgetary approach, or a needs-based budget. The perceived goal is to stop basing future budgets on past budgets, and instead form budgets on new, or re-stated priorities and needs.

Recurring journals subscriptions are directly impacted by these changes and thus the library’s Journals Review Committee (JRC) needed to respond proactively and constructively to develop new materials to support communication with the university and library administrations. The JRC is responsible for a little over half of the library’s collections budgets and thus needed to be prepared in case a more detailed justification was required for our budget requests.

The CMT team is a subgroup of the JRC and provides collection analysis reports and tools that support the committee’s work. The Budget Proposal drew on previously developed reports to support the JRC’s bi-annual collection review process. These reports included both the Collection Review and the Package Level Analysis report (PLAR), each of which supported decisions to implement strategic cuts and stay under budget. Unlike the Collection Review report and the PLAR, which focus on cuts, the goal for the Budget Proposal was to create a new report that focuses on what the library needs to keep. This new report is intended to showcase the value provided by the library’s journals and journal packages. Over the last decade the JRC cut hundreds of subscriptions and numerous journal packages to respond to flat budgets and journal inflation. The Budget Proposal allowed the committee to flip the script and focus on positive arguments to keep our journals and journal packages.

Gustafson-Sundell reported that the Budget Proposal served as an umbrella project for several sub-projects. Work on the report supported the development of new Key Performance Indicators, or KPIs, for the library’s journals and journal packages. These KPIs were the focus of a summary sheet on the report called the Budget Proposal Overview. The Overview also links out to Focus Reports for each package that encouraged the CMT team to explore new instruments for communication, including novelty data visualizations. As the CMT discussed ideas for these KPIs, colleague Heidi Southworth proposed an idea for a whole new category of metrics called Subject-Package measures. These Subject-Package measures supplement the Budget Proposal and became a new collection analysis tool for the JRC.

Previous Reports

Gustafson-Sundell demonstrated the Collection Review report, the Package Level Analysis report (PLAR), and the Collections Power BI (CPBI) that all feed into the Budget Proposal.² The Collection Review report was developed in 2017. It provided a list of all the library’s individually subscribed journals and all titles in journal packages. The Collection Review matched each title in the list to data from a variety of sources. The report included data from the library’s integrated library system, COUNTER usage data reports, vendor pricelists, Scimago Journal Rankings (SJR), interlibrary loan, and more. The report included over one hundred data variables, including calculated metrics and holdings analysis. In addition to allowing individual title analysis, the Collection Review report rolled up individual title data for summarization at the package level.

The PLAR was developed in 2019. The PLAR simplified the collection review process by providing a clearer comparison of the differences between journal packages. In developing the PLAR, the CMT team created new KPIs. Gustafson-Sundell highlighted one example of a KPI developed for the PLAR that was especially useful for identifying potential cancellations, the Sub Usage Ratio. The Sub Usage Ratio indicated how much usage is specific to the subscription platform, so it is a quick way to see the impacts of coverage overlaps with other sources of access.

Gustafson-Sundell reported on the CMT team’s experiments with data visualization based on both the Collection Review and PLAR. The team used Tableau briefly but moved onto Python to facilitate automated report production. Eventually they settled on Power BI as the most effective tool for data visualization. Power BI was useful because reports can be shared across campus online and their administration has familiarity with using the tool.

Gustafson-Sundell demonstrated a third report that feeds into the Budget Proposal, the CPBI. The CPBI is an interactive dashboard of charts and tables, available online across campus. He explained that the CPBI is primarily used as an outreach tool, so that librarian liaisons can communicate effectively about collections to academic departments. The CPBI is used to support accreditation reviews and collection review decisions. The current iteration of the CPBI includes eleven pages of charts and tables. Most pages are interactive, primarily by using filters.

Budget Proposal and Key Performance Indicators

Evan Rusch presented the Budget Proposal Overview, describing each of the Key Performance Indicators in the report. He reiterated that the purpose of the Budget Proposal is to flip the script from focusing on cutting the weakest of the library’s journal packages, to emphasizing the value of each journal package and demonstrate the unique attributes that each provides.

Rusch provided background for thinking about the KPIs they had developed as a part of the Budget Proposal. One goal was to emphasize that there are challenges for evaluating journal packages exclusively via cost-per-use (CPU), which provided a justification for developing the diverse group of KPIs that are a part of the Budget Proposal. Rusch described three categories for assessing how collections add value for the campus. Each of these categories are represented within the KPIs. One category for assessing value in a journal package is supply, or the amount of content a package provides the university or for a given subject area. The number of journal titles and the number of citable documents published in a journal are examples of supply metrics. The reports also use quality as a way of assessing value. This metric looks at the amount of high-quality content that a package adds to overall holdings. Examples include established citation metrics and—specifically for the CMT team’s work—the SJR, because the data was accessible to the group. A third category for assessing value in a journal package is usage. This looks at the amount of usage that is provided by content in each package. COUNTER usage statistics are the primary data source for this category, but other data, such as link resolver data or interlibrary loan data, can also provide useful usage metrics. Rusch explained that each of these categories needs to be tempered by package cost. As a result, the KPIs incorporate cost into metrics utilizing the three value categories.

Budget Proposal Overview

Rusch described the parts of the Budget Proposal Overview (see Figure 1). The overview is an Excel spreadsheet with each row representing journal packages and the columns representing KPIs or other relevant data. The report utilizes conditional formatting to highlight positive numbers in various KPIs keeping the focus on attributes that justify a package’s subscription.

The initial column of the Budget Proposal is cost, followed by the number of journal titles in a package. Providing this basic metric for supply provides context for the cost since administrators may have no sense of the size of any of the packages. The first KPI listed in the report is Sub Usage Ratio. The report also includes columns for the two sources of data that determine this metric. The first of these is for usage that comes directly from the journal package’s platform, which is called Sub Downloads. The subsequent column lists the number of Total Downloads, which adds usage that has come from aggregator access to the Sub Downloads number to give a total number of uses. Sub Usage Ratio describes what percentage of total downloads came directly from package’s platform. The report uses the last three years of data for these metrics to balance out one-year spikes or dips. By drawing attention to journals that have high Sub Downloads, Sub Usage Ratio demonstrates when full-text aggregator access may not be sufficient.. High Sub Usage Ratios suggest that a package likely provides content exclusively available from the publisher and may leave interlibrary loan as the only access in case of cancellation.

The next KPIs pull their concepts from the three value categories of usage, supply, and quality. The first, Utility Value Indicator, is a variation on cost-per-use utilizing three years of usage data divided by one year of cost. The second, Supply Value Indicator, compares a package’s supply of content to its cost. This divides the subscription cost of the package by the number of citable documents produced over three years. The number of citable documents is a metric used to create SJR and is a basic number to represent the count of articles produced by a journal. That number is then aggregated to provide the total of articles published in a journal package over three years. Supply Value Indicator is useful for assessing whether the library receives sufficient content for the cost of a package.

The third of this set of KPIs is called the Supply-Quality Value Indicator. The Supply-Quality Value Indicator looks at the cost per article supplied for those journals that are in the top quartile of Scimago Journal Ranking. These are the most heavily cited journals within their respective disciplines. This creates a metric for the supply of high-quality content, and allows for a comparison with the previous KPI, Supply Value Indicator. For some packages, there is very little difference between these two KPIs meaning that almost all the articles supplied in the package are from highly cited journals. Conversely, if a package is filling out its holdings with less cited journals, it might dampen the impact of a high Supply Value Indicator number for a package. Rusch suggested that this quality-focused metric can be emphasized in accreditation visits or to promote graduate programs to their perspective students.

The next KPI in the report tracks whether usage is trending up or down for a given journal or package. This number, called the Southworth Ratio, divides the last four years of usage by the last eight years. If the number is above 0.5, usage is trending upwards, if it is below 0.5 it is getting less use than in the past. This allows the library to positively show which packages have seen increased use.

The Budget Proposal Overview utilizes year-of-publication usage data in the next set of KPIs. These numbers provide data for the publication year of downloaded articles. The report offers three different ways to look at year-of-publication data. The first divides the number of articles retrieved from the last two years of publications by the number of articles retrieved overall. The higher the percentage, the greater the usage is coming from the most recent articles published. Along with our Sub Usage Ratio this can provide insight as to when aggregator access would not be a good alternative as recent content might be embargoed in aggregator access. Beyond content being replaceable, Rusch suggested that when recently published articles have high usage, it is important to have up-to-date current subscriptions. Additionally, the report includes year-of-publication metrics for the percentage of downloads from the most recent six years. The six-year figure aligns with longer embargoes, such as exist in some JSTOR collections. Lastly, the report provides the percentage of pre-2000 downloads for a journal package.

The last KPI included in the Budget Proposal Overview provides the supply of high-quality journals in a journal package by looking at the number of Scimago Journal Rank Quartile one citable documents. The Q1 Journal Ratio measures the percentage of citable documents in a package that come from Quartile one journals.

The KPIs provide differing focus points for advocating to maintain or even increase our subscription budget. Sub Usage Ratio gives a sense of replicability or uniqueness in access that a package provides. Utility Value Indicator, Supply Value Indicator, and Supply-Quality Value Indicator provide opportunities to weigh package cost against our three categories of value, usage, supply, and quality. Southworth Ratio allows one to understand usage trends. Year of Publication data provides insight on replaceability, but also expresses the value for current research. Lastly, the Q1 Journal Ratio measures the amount of high-quality content within a package.

Subject Package Measures

The Budget Proposal Overview provided metrics that demonstrated the general value of packages, but the KPIs do not address the value of a package to a specific subject area. Rusch explained that Subject Package Measures create a simple table that demonstrates which packages are important to a given discipline and then similarly show how a journal package’s usage or supply was distributed amongst various subject areas (see Figure 2). These subject-oriented measures provide another data point for advocating for collections.

Rusch demonstrated an example of a Subject Package Measure report. The report divides numbers for usage or supply across the Scimago subject areas to demonstrate how each journal package adds value for that discipline. The rows of the table are Scimago subject areas, and the columns are journal packages. In the example, the table employed usage data, where the number listed is the percentage of usage in a subject area that comes from a given journal package. This shows how important a package is for a given subject.

Another way of using this table to advocate for journal packages is to demonstrate how many subject areas rely on a general package. The version of the table demonstrated used conditional formatting to highlight cells where a package provides 10 percent or more of the usage for a subject area. A few large, multi-subject packages were shown to provide 10 percent or more of the usage for numerous subject areas. The Subject Package Measures report demonstrated that some subject-specific packages provide a depth of coverage for their respective disciplines, while some general packages have an impressive breadth of subject areas they serve. The CMT team generated similar tables for metrics other than usage, such as the number of citable documents supplied and the number of Q1 journal titles for a subject area. Rusch explained that any data points from the Subject Package Measures report can be incorporated into the Budget Proposal to advocate for a given journal package.

Package Level Focus Reports

Rusch described Package Level Focus reports. Linked from the Budget Proposal Overview, these reports focus on specific packages. They pull positive metrics from the budget overview and Subject Package Measures reports into a quick guide. The goal is to advocate for journal packages and provide a contrast to the spreadsheet approach of the Budget Proposal Overview. The Package Level Focus reports have some uniformity, but ultimately, they reflect the unique contributions that each package provides.

Package Level Focus reports use Microsoft PowerPoint slides, which help create a visual and memorable way to see the attributes of a given package. PowerPoint was chosen because it is an effective tool for demonstrating visuals, and slides can easily be incorporated into other presentations.

Rusch demonstrated a report for a large multisubject package. The first slide displayed visualized data from the CPBI that showed the supply of journal titles for various subject areas. The chart included a breakdown of those titles by SJR Quartile within each subject title list. The second slide in the report featured the Subject Package Measures report and further demonstrated the breadth of subject areas this package serves. The remaining sections of the Package Level Focus report highlighted individual KPIs or other attributes for a package, accompanied by an image generated by Open AI’s Dall-e 3. Rusch emphasized that the goal of these images is to create a metaphor or memorable view for the audience. The CMT team has developed a small library of images to use in multiple Package Level Focus reports or other presentations. One example is an image of a dragon sitting on a pile of gold, which emphasizes the dragon as an expensive package and the gold as a pile of riches the package provides (Figure 3).

Figure 3.

Example Slide from Package Level Focus Report

To provide a contrast of the multisubject package, Rusch demonstrated a second Package Level Focus report for a subject-specific journal package. He emphasized that at MNSU many subject-specific packages do not seem as strong when assessing based on cost-per-use. He provided statistics that showed the depth of support this package provided for the specific discipline it served, and shared a testimonial from an MNSU professor about the package’s essential role in supporting teaching and learning within their academic program. He also shared that the package had a high Sub Usage Ratio, meaning the content was likely not replaceable through other sources and is directly connected to accreditation efforts.

In each case, the Package Level Focus report emphasized the unique attributes that made that package important for MNSU by pulling together data from the CPBI, Subject Package Measures, the Budget Proposal Overview, and our developing image library to create a positive impression of the package.

Subject Level Brief Report

Pat Lienemann explained that the audience for the Budget Proposal was primarily administrators and others most concerned with library budgets. Lienemann reported on a project that colleague Heidi Southworth conducted, which translated aspects of the Budget Proposal into a report intended for academic departments. Lienemann demonstrated a Subject Level Brief report to be shared with the Electrical Engineering department at MNSU and possibly be used during accreditation meetings for that program.

In communicating with faculty, library staff focus messaging on the accessibility of content to support teaching and research rather than the value gained for the cost of a package. Thus, an important component for the Subject Level Brief report is to compare the library’s holdings to all publications that support electrical engineering. Scimago Journal Rankings group journals into 27 subject areas and then 309 narrower subject categories. Under the engineering subject area, Scimago’s subject category of Electrical and Electronic Engineering provides an excellent comparison for the library’s electrical engineering journal holdings.

Lienemann emphasized that the Subject Level Brief report utilized the same categories of value as the Budget Proposal, usage, supply, and quality, as well as cost, even if it is of reduced interest to this audience (see Figure 4). The report highlights MNSU’s journal supply in the Electrical Engineering subject category and considers that supply by SJR quartile. Lienemann described a chart that shows the percentage of access the library has for electrical engineering journals for each SJR quartile, demonstrating that the library provides access to 63 percent of the journals in this category. The goal for the report is to assure faculty and accreditors that the library provides sufficient journal coverage for electrical engineering.

Figure 4.

Categories of Value

The chart also shows the library’s level of access based on the quality of the journals or their SJR rank. The chart demonstrated that the library’s holdings are stronger for the higher-quality journals. Lienemann noted that MNSU provides access to 91 percent of top-quartile electrical engineering titles. The report also provides usage information for individual journal titles, both the number of downloads and the Southworth Ratio, which shows how usage is trending.

Lienemann then described the next chart included in the report, which shows downloads in 2022 by year of publication. Year of publication usage data helps inform the library if journal usage is coming from recent or older publications and can help faculty understand how their students are using campus resources. The chart also breaks down usage by journal package. This chart demonstrated that 65 percent of the 2022 downloads for this subject came from a subject-specific package highly valued by the faculty and accreditors. This helps make the case for maintaining packages and reassures the faculty that their valued resources are being used. Additionally helpful is that a multisubject package accounted for an additional 23 percent of the downloads, reinforcing this package’s value to their program.

The report goes on to address cost, helping faculty to compare packages that are important to them. Lienemann explained the next page in the report, which showed both the cost-per-use and the cost-per-article supplied by the subject-specific and multisubject packages that had high usage on the previous page of the report. This example illustrates that even though the subject-specific package is important for their discipline, it still has a higher cost-per-use and cost-per-article than a major multisubject package. This information can help them understand the challenges librarians face when advocating for valued content.

Lienemann emphasized that while sharing data with faculty members is important for their knowledge of library collections, this process also communicates the library’s strategy for advocating for their programs. Thus, the Subject Level Brief report is not just a presentation to a department, but a tool to improve discussions about library collections and collection budget.

Lienemann suggested points to consider when communicating with faculty on campus. He suggested that librarians remind faculty that library resources are a part of the academic ecosystem and information from academic departments can help to better support them. He encouraged attendees to solicit feedback from faculty to improve arguments for justifying collection budgets. Lienemann concluded by highlighting the next steps for this project, which will focus on sharing the Budget Proposal with MNSU library’s dean and other library colleagues to get additional feedback. The CMT will continue working with faculty to improve the library’s advocacy for its collections.