1 Introduction

Community-based research connects faculty members, postsecondary students, and community partners for the purpose of examining and addressing a community-identified need (Strand, Marullo, Cutforth, Stoecker, & Donohue, 2003). Community-based research is expected to innovatively address social issues as relevant knowledge is generated alongside actions that actively engage local people (Janzen, Ochocka, & Stobbe, 2017; Savan, 2004), yet traditional power dynamics can subordinate the needs of the community below the needs and interests of researchers (Bringle & Hatcher, 2002). Various partnership models exist to facilitate relationships between community-based organizations and campus researchers to help create mutually beneficial research goals. However, community-campus partnerships have historically privileged the needs of universities and students seeking work-integrated learning and skills over the immediate research needs of community-based organizations (Bortolin, 2011). Thus, there is a need to refocus on community outcomes and impacts that arise from community-based research and community-campus partnerships.

The central question of this article is how the projects and processes of the U-Links Centre for Community-Based Research (U-Links) have contributed to social change within Haliburton County, Ontario. The research evaluates the experiences of local organizations who have hosted community-based research projects facilitated by U-Links by exploring the perceived contributions of these research projects to capacity building and social change in the community.

The article begins with a discussion of the role of community-based research brokers and situates U-Links within the broader community-campus partnership landscape. We then discuss the challenges in robustly evaluating the impacts of community-campus partnerships on the community. Our research approach and methods are described, including our approach to evaluating the impact of U-Links in the community, and our main findings are presented. Finally, we discuss some conceptual contributions of the research.

1.1 Brokering Community-based Research

Despite a large body of literature regarding the values and ethics of community-based research, practitioners often find it challenging to “walk the talk” (Stoecker, 2009). Communities may benefit less than the academics and students, despite the desire for reciprocal and mutually beneficial outcomes (Alcantara, Harper, & Keys, 2015). Furthermore, communities and academics have different resources, jargon, expertise, expectations, and timing that can hinder research partnerships (Tennyson, 2005). Because of this, community-based research partnerships often rely on a facilitating organization that acts as a translator or “broker” to help establish and manage the relationship (Tennyson, 2005).

Levkoe and Stack-Cutler (2018) define research brokers as individuals or organizations that “act as intermediaries between community-based organizations and academic institutions with an aim to develop collaborative and sustainable partnerships” (p. 20). Brokers build connections, support relationships, and share knowledge. These formal or informal arrangements take on a variety of structures and strategies depending upon funding, governance, stakeholders, organizational activities, strategic mandates, or broader motivations (Levkoe & Stack-Cutler, 2018). Brokers can play a role in community engagement, supporting partnerships and project development, management of projects and partnerships, research dissemination, and developing next steps (Levkoe & Stack-Cutler, 2018). Brokers are the catalysts for building relationships.

2. Research Context

2.1 U-Links Context and Organizational Approach

U-Links is a community-based nonprofit organization that brokers research between local postsecondary institutions and organizations across Haliburton County, a rural region in central Ontario, Canada. Haliburton County is a 4,000 km2 region within the Canadian shield with a year-round population of about 18,000 (Statistics Canada, 2016). In the summer, this grows to roughly 45,000 people due to seasonal cottagers and tourism. There is a high number of seniors relative to the provincial average, and the local economy is dominated by serving the needs of the seasonal residents.

For almost 30 years, U-Links has helped roughly 90 different local organizations to develop, manage, and retain nearly 450 community-based research projects completed by students. U-Links is an independent organization that operates under the auspices of the Haliburton Community Co-op, receiving funding from Trent University, local organizations, and the Township of Minden Hills. U-Links independence from the university is unique within community-campus partnerships. Levkoe and Stack-Cutler (2018) promoted the merits of this brokering model due to its structural commitment to community rather than campus priorities. However, they also noted the importance of a predictable source of funding for brokering organizations, which can be a key limitation for a nonprofit organization like U-Links.

Research partnerships brokered by U-Links involve four actors: a project host representing the community organization, a faculty member (typically but not always from Trent University), student researcher(s), and a project coordinator from U-Links. Each year, U-Links issues a call for research within the community and organizations respond with their needs. The U-Links project coordinator and community organization then develop these projects into proposals that appear on U-Links’ website for prospective senior undergraduate (and sometimes graduate) student researchers. Some students apply to a research project that they complete as a stand-alone community-based research course, while others are matched through a course that has a community-based research component built into the syllabus. Once the project has been matched and formalized through a project agreement, the U-Links project coordinator steps back from driving the process, allowing the students and host to take the lead in completing the project. U-Links staff will step in if there are conflicts or when additional support is warranted. At the end of the project, the host receives the final report, and the student receives a grade from their professor. U-Links disseminates the research through an annual celebratory event with the community and in an online database accessible to all.

2.2 Challenges evaluating the impacts of community-based research.

It can be difficult to show causality between a community-based research project and a particular outcome. Community-based research projects lead to a diversity of outcomes including public installations, new organizational protocols or programs, increased organizational capacity (in the form of added human resources), or expanded social networks (Minkler, Vásquez, Tajik, & Petersen, 2008). Outcomes can also be abstract such as an increase in empowerment or shifts in behaviour (Middlemiss, 2011). Thus, evaluations of individual projects are often complicated and contextually bound.

Evaluations that explore the broader community-scale impacts of community-campus partnerships are rare. This is likely due to the challenges of explaining how these projects and partnerships lead to specific outcomes within social systems (Sandy & Holland, 2006). Causality is a confounding factor in impact evaluations as community-based research (and associated partnerships) are only one part of a larger socioeconomic, cultural, political, and environmental system (White & Phillips, 2012). Evaluation theorists address these issues of complexity by distinguishing between the attribution and contribution.

Attribution describes the cause of an effect and estimates how much of that effect is due to an intervention through statements of outcome and strength of outcome. Contribution suggests that an intervention has made an observable difference but recognizes that multiple acting forces can influence an outcome (Mayne, 2012). Attribution implies direct causality, whereas contribution reflects a more complex social landscape (Mayne, 2012). John Mayne’s (2008) contribution analysis evaluation framework suggests a set of requirements for establishing meaningful contribution claims when exploring cause and effect, including:

  1. The program should be based on a theory of change with reasonable and agreed upon assumptions about why the program is expected to work.

  2. The activities of the program have been implemented.

  3. The theory of change is verified by evidence showing that the chain of expected results occurred.

  4. Other factors influencing the program were assessed and were either shown not to have made a significant contribution or, if they did, their relative contribution was recognized. (Mayne, 2008)

When other influencing events and conditions muddle a program intervention, contributions may be found to be neither necessary nor sufficient to create observable social outcomes on their own (Mayne, 2012). By following a contribution analysis methodology, and using the language of contribution rather than attribution, the role of program interventions can be highlighted as a supporting force for change, without overstating or understating their impact (Mayne, 2008).

Another challenge is that communities and academics may have different objectives in evaluation. Evaluations of community-based research in the academic literature tend to focus on its effects on students rather than the community (Cruz & Giles, 2000). Although communities may also take part in evaluating community-based research projects, these evaluations may be motivated by the expectations of program funding rather than academic learning. Furthermore, communities are more likely to communicate about their evaluation practices to their own local community (or internally) rather than in academic journals. This can make finding examples of community-driven community-based research evaluations particularly difficult for academics.

Moreover, community-based research does not always create positive outcomes for the community. Community-based research can be a drain on community resources; partners involved may not adequately address issues of scope, expertise, or time constraints; and intrapersonal relationships may break down (Sandy & Holland, 2006). These negative experiences may reduce the willingness of some community partners to participate in an evaluation process.

2.3 Addressing Challenges through Collaborative Evaluation.

Strand et al. (2003) offer a number of principles for encouraging a healthy community-based research partnership:

  • a shared worldview;

  • agreed upon objectives and approaches;

  • a relationship built upon mutual trust and respect;

  • shared power, reciprocal communication;

  • empathy and flexibility; and

  • a commitment to work to ensure:

    • the partner’s priorities have been met,

    • the project has enhanced organizational or community capacity,

    • and the partnership shares responsibility to mobilize long-term change. (Strand et al., 2003, paraphrased from pp. 8–9)

Any evaluation of community-campus partnerships should also be a collaborative and equitable endeavor (Sandy & Holland, 2006). By identifying and amplifying how outcomes are envisioned by community members, and using these as the foundation of the evaluation, we can examine the impact of community-based research for those whom it was meant to affect (Reynolds, 2014). The evaluation process should both reaffirm the explicit community-first values of the partnership and actively address conflicts that may arise (Marullo et al., 2003). If community-based research is itself a reflective process that sheds light on different perspectives and realities within the community, evaluations should also be approached as inherently reflective, critical, and perhaps even humbling. These values guided our research regarding the contributions of U-Links within both the organization and the broader community.

The research builds on Jagosh et al. (2015), who first looked at a realist evaluation of community-based research, and on modern evaluations theories such as nested actor-based theories of change (Koleros & Mayne, 2019) and linked contribution and realist approaches (Whynot, Lemire, & Montague, 2018), exemplifying the importance of both relationships and resources in enabling community-based research mobilization and supporting the findings of Balazs and Morello-Frosch (2013). We feel that evaluations approached from a participatory mindset and motivated by organizational growth rather than funder-imposed accountability measures can address many of these challenges.

3 Research Approach and Methods

Our approach uses contribution analysis and realist evaluation as layered tactics within one program evaluation. Our evaluation approach was selected using criteria developed by Befani (2016).

Contribution analysis brings together evidence about the outcomes of a program and relates this back to an existing theory of change (Mayne, 2008). Realist methodology asks, “What works, for whom, in what respects, to what extent, in what contexts, and how?” (Pawson & Tilley, 1997). Realist evaluators aim to understand the mechanics left by contribution analysis and why they had an influence in that context (Astbury & Leeuw, 2010). In particular, realist evaluations result in a series of context-mechanism-outcome statements (Westhorp, 2014). Contribution analysis and realist evaluation can coexist as layered approaches within one program evaluation, with the former providing broad strategic learning about implementation and the latter focusing on smaller and more promising contexts required in the embedded program theory (Blamey & Mackenzie, 2007; Koleros & Mayne, 2019).

These two approaches are linked in that they both rely on an underlying theory of change model (Leeuw, 2012). A theory of change is an explanatory model that identifies how a social intervention or program is expected to contribute to a logical chain of intended outcomes (Funnell & Rogers, 2011). These outcomes can be short- or long-term objectives and may affect a person, group of people, or the community at large (Janzen et al., 2017). A theory of change is context specific to an organization, or to a particular program within an organization, and is meant to reflect the initial aspirations of the organization rather than the realized outcomes (Valters, 2014).

For a contribution analysis, Mayne (2012) argues that a reasonable claim about a contribution can be made if the following three criteria are met:

  1. The original theory of change is logical and plausible, supported by research or existing evidence.

  2. The activities set out in the theory of change were implemented as described, and the chain of expected results occurred.

  3. Additional influencing factors were either minimally significant or their role has been recognized.

In our research, contribution analysis explored the implementation and outcomes of the program, and the realist evaluation explored the contexts necessary for the program theory to hold true. Contribution analysis allows us to establish “what is expected to happen” (the initial theory of change) and “what appears to have happened” (the contribution narrative and claim). The initial theory of change can then be adjusted or confirmed using empirical evidence and arguments:

  • Does the original logic hold true based on the data collected?

  • What needs to be adjusted within the theory of change to better represent the contribution pathways required to achieve desired program impacts?

  • What is the significance of outside factors?

Then realist evaluation approaches (what works? for whom? when?) were developed to consider the contexts and mechanisms that influence various contribution pathways. These are presented as a series of context-mechanism-outcome statements and give depth and specificity to strengthen the logic of the contribution analysis.

To conduct our evaluation, the sequence of our research activities was as follows: first, a theory-of-change workshop was conducted with the U-Links Management Committee in February of 2018; next, interviews were conducted from June of 2018 to August of 2018; participant observation took place throughout the research process. The research approach was approved by the Trent University Research Ethics Board and by the U-Links staff and Management Committee.

3.1 Data Collection and Analysis

Primary research was conducted through a preliminary theory of change workshop in February 2018; semi-structured interviews with U-Links Management Committee members, past program administrators, and project hosts; and completion of a research impact template as part of each interview with project hosts.

3.1.1 Theory of change workshop.

A theory of change workshop with the U-Links Management Committee was conducted to identify U-Links’ activities, outcomes, and indicators of success. The workshop used a modified “tree of means-and-ends” from Chevalier and Buckles (2013) to map the theory of change. A tree of means and ends is a participatory tool of mapping out ideal outcomes (i.e., the ends or vision, see Figure 1) along with the causes and effects (Chevalier & Buckles, 2013). The exercise helped to identify clear and manageable goals and a strategy for how to achieve them (Chevalier & Buckles, 2013).

Figure 1.
Figure 1.
Figure 1.

Photo of the visioning exercise at the theory of change workshop.

Participants worked in groups of two or three to discuss the different activities U-Links engages with (represented as roots on the tree), short-term outcomes of these activities (branches), and indicators of success (fruits); each aspect was assigned a different color of sticky note to provide clarity.

Each group posted their sticky notes onto a poster board with a pre-drawn tree stencil (Figure 2). After 10 to 15 minutes of discussion, when the addition of new ideas to the tree had slowed, each of the sticky notes were read aloud and then sorted by the group into similar categories, making sure there was general agreement about how the responses were grouped and organized among the roots, branches, and fruits. This process of “free-listing” and “pile-sorting” (Chevalier & Buckles, 2013) is a consensus-based method of generating, consolidating, and clarifying ideas.

Figure 2.
Figure 2.
Figure 2.

Pile sorting features on the tree of means and ends at theory of change workshop.

After each section had been confirmed, outcomes, indicators, and activities were rearranged into sets, forming a logic model that traced the pathway between an activity, the outcome of that activity, and an associated indicator of success. In some situations, additional sticky notes were added to complete the logic grouping.

A few weeks after this workshop, a conceptual representation was produced to describe U-Links’ theory of change and sent back to the Management Committee for edits, final considerations, and additions. This final consultation clarified items and expanded on previous themes that arose during the workshop. The theory of change allowed for the goals and reasoning of U-Links to be addressed and analyzed as a baseline for the evaluation.

3.1.2 Interviews.

Beginning in June 2018, a total of 26 interviews were conducted with past project hosts, municipal representatives, and U-Links staff and Management Committee members. A recruitment email was sent to contacts from 98 past project host organizations by the executive director of U-Links. Snowball sampling was an additional source of recruitment.

The interviews helped to understand how projects were used by the community organizations and their perceived impacts. The interviews with project hosts were guided further by a Research Impact Assessment tool and Research Engagement Assessment tool developed by Research Impact Canada (D. Phipps, personal communication, March 2018). These tools provided interview questions to explore the contexts and mechanisms that contributed to the perceived outcomes of individual projects. Several of these project examples were followed up by collecting evidence of impact from other sources (e.g., the research report). Multiple sources of evidence also help triangulate the data and minimize subjective bias (Jonsen & Jehn, 2009).

Transcribed interviews were analyzed with NVivo 12 software to identify thematic contribution narratives for U-Links’ key stakeholders and establish context-mechanism-outcome configurations. A set of indicators were developed to assess whether the chain of events described by interview participants matched the expected activities and outcomes outlined in the theory of change and the degree to which causal link assumptions were supported by the data.

4. Findings

4.1 Theory of Change

Figure 3 illustrates U-Links’ theory of change, revealing their logic and assumptions regarding community-based research and impact in Haliburton County and providing the baseline indicators necessary for the evaluation. U-Links’ theory of change was developed from the workshop and draws inspiration from Morton (2015) and Koleros and Mayne (2019).

Figure 3.
Figure 3.
Figure 3.

U-Links theory of change for community-based research.

Figure 3 shows the four impact pathways stemming from the interactions of the U-Links program for each of the key actors: local governments, postsecondary institutions, the public, and host community groups. Although U-Links engages with both community groups and local governments in similar ways, their level of influence over countywide changes are distinct. Each branch of the theory of change can be thought of as its own individual contribution pathway, with outcomes interdependent of one another and processes that may take place consecutively or concurrently depending on the situation. Each activity is intended to lead to outcomes that can be assessed by indicators.

4.2 Characteristics of Community-Based Research That Benefit Community Partners

Although the assumption is that the outcomes of a community-based research partnership will be, at minimum, sufficient to justify the time and energy invested by all parties, we know this can’t always be the case. So, what is it that makes a community-based research project successful?

Interview participants were asked how they would describe both successful and unsuccessful community-based research projects to identify how and why project outcomes are deemed successful or unsuccessful from the perspective of project hosts. Traits of successful and unsuccessful projects as described by interview participants have been summarized in Table 1, broken down by research partnership, research quality, and resources available. Most interview participants spoke about unsuccessful projects in terms of the experiences with students rather than the products of the research, perhaps recognizing that building relationships was also a desired outcome.

Table 1. Traits of Successful and Unsuccessful Community-Based Research
Theme Positive outcome Negative outcome
Partnership
Community engagement Community/key stakeholders were excited about and active within the research process Community members felt excluded from or disinterested in the research process, student didn’t come up to the rural Canadian community
Relationships within project team Students and hosts had strong, ongoing communication; “project runs itself” Students and hosts had a breakdown in communication/trust, internal conflicts
Student passion Students became inspired and excited about the research project; “student pulls themselves into the project” Students viewed project as “just another assignment,” were disinterested
Student growth & learning Students gain new skills and share positive experiences from the project, hosts report pride in mentorship Students do not gain new skills and are unable to complete project effectively (or at all), host feels their time or investments weren’t respected
Research quality
Standard of rigor Student skills were appropriately matched to the project, initial questions were answered, high-quality work Students were unable to complete the project to the necessary standards to be “useful,” poor quality work, questions left inadequately answered
Research utility Project was relevant to the organization’s needs and able to be used with minimal adjustments, lives up to initial goals and informs organization, feasible recommendations Project lost focus and outcome was not relevant to the community group, does not provide new knowledge, irrelevant or unfeasible recommendations
Scope & depth Scope and depth of project was appropriate to the time line and skills, goals were achieved Mismatched project goals and time line/abilities, final report lacked important depth
Resources
Resource availability Students, hosts, and faculty had the time, skills, tools, and information necessary to support the project Students, hosts, and faculty had insufficient time, skills, tools, or information necessary to support the project

Interview participants identified four partnership characteristics that can affect a community-based research project: community engagement, relationships within the project team, student passion, and student growth and learning. Community engagement refers to the level at which key stakeholders were included and interested in the research process. Projects where outside community members were minimally involved or consulted were defined as less successful than those that actively engaged the public. Relationships within the project team (which includes the community host, student researcher[s], supervising faculty, and a project coordinator) were also a key factor in defining the success of projects. A breakdown between any of these actors can trigger an untimely end to the partnership and overall poor-quality research outcomes. Conversely, when these relationships thrive, organizational hosts describe the project “taking on a life of its own” and “running itself,” requiring little oversight or check-in from the project coordinator. These relationships tended to be described as reciprocal and respectful, with ongoing communication, bound by clear expectations from one another (as defined in the project agreements), and a shared vision for the project.

Student passion was identified as a vital element of the research partnership for successful projects. Their level of engagement with the project led to host satisfaction or frustration. Some interviewees felt that some students see community-based research as “just another assignment,” and these attitudes reflect poor commitment to the project. Finally, student growth and learning were identified as a key element in the success of these partnerships. Hosts shared a feeling of empowerment as mentors and guides for the students taking on their projects. When students came away from a project with new skills, project hosts reported satisfaction and success, whereas when students faltered (and hosts felt they had invested time and resources to help), hosts felt their time or efforts were disrespected and thus reported poor research outcomes.

A final product should be high quality in order to meet the needs of project hosts. Factors relating to research quality include the rigor of the research itself, the utility of the research, and the scope and depth of the project. Project hosts reported positive outcomes from research in which the student skills were appropriately matched to the initial research proposal. Poor writing or lack of experience with data analysis were common concerns among participants, which led to the characterization of projects as being less rigorous. Participants noted projects needed to be well defined in terms of scope and depth and strategically developed to meet organizational needs to provide positive outcomes. The timing of the academic year can constrain the scope of projects, or student skills may not be sufficiently developed to tackle politically and socially sensitive topics or higher caliber research. The scope, depth, skills, and subject matter of a project should be aligned with the student’s capabilities and constraints in order for the project to be successful.

The level of resources available can make or break a project. Resources required for community-based research projects include time, funding (to support students), access to materials (such as past research or equipment), and access to individuals (knowledge holders or key stakeholders). U-Links was often cited as being instrumental in the “creative” acquisition of resources, whether through their networks or collaboration with the host or faculty involved.

4.3 Developing and Supporting Projects with Community Benefit

Not all projects are designed to have direct impact. U-Links projects broadly fit into two categories: “one-and-done” projects, which address an organization’s specific decision or course of action, and “strategic” projects, which encompass longer term considerations across the region. While all projects have their own specific circumstances and goals, one-and-done projects have been a way of scaling up the number of U-Links projects with limited staffing. In these projects, student(s) typically complete a literature review, ask their community group or target audience a few questions, and then provide some recommendations within a final report. The host then either moves forward or changes course based on recommendations in the final report. These projects are expedient for both the student and the host and can be completed in a single academic term.

In contrast, more strategic projects involve dividing a larger topic into a collection of smaller projects hosted by one or more community organizations. These projects can either be sequenced (one project informs the next) or conducted concurrently to provide a more complete set of information on a complex topic. The host(s) use these projects to network outwardly, speaking to and sharing this information with other organizations, or pursuing action at the municipal level. These distinctions tend to speak to the host’s capacity in terms of strategically employing the projects or the time and resources to consider implications broader than their own organization.

One-and-done projects are not necessarily perceived as any less beneficial to project hosts if they are completed competently, but in terms of leveraging the work that’s been accomplished, longer term strategic projects tend to have a greater impact on Haliburton County in that they nurture institutional and organizational partnerships within the community. These partnerships appear to drive future research projects and continued community engagement long after the initial research project has concluded. These spin-offs and social network changes create cyclical ripple effects flowing out from community-based research projects. The premise of the ripple effect concept is that community-based research activities are a series of “events in the history of a system, leading to the evolution of new structures of interaction and new shared meanings” (Hawe, Shiell, & Riley, 2009, p. 267). The ripple effect is a model to help understand how partnership activities build between stages, with the outcomes of one stage of the partnership informing or transforming the context of subsequent stages (Jagosh et al., 2015).

Past project hosts were asked to describe the factors they felt were most important to developing successful research projects. These factors are summarized in Table 2, categorized into three areas: partnership design, research design, and resources. The factors provide some essential contexts and mechanisms that enable U-Links projects to be successful, though they do not guarantee it.

Table 2. Factors Important for Developing Successful Projects
Partnership design features

  • Equal and committed relationship between students and hosts

  • Mutual learning, mutual benefit, reciprocity, respect

  • Openness, flexibility, trust

  • Clearly defined roles and objectives (project agreement)

  • Meetings/communication between U-Links, student(s), and host (face-to-face interactions)
Research design

  • Project motivated by larger strategy/immediate need/window within community/organization

  • Realistic scale/scope of projects (awareness of timelines and capabilities of undergrad students, involvement of graduate students where necessary, sequential and higher-level research projects)

  • Concrete measurements of success and project deliverables/timelines (project agreement)

  • Community engagement embedded within project deliverables/design
Resources

  • Host ability to network and share results of projects

  • Partnership funding/resources to support project

5 Discussion

This evaluation set out to answer whether U-Links projects and processes have contributed to social change within Haliburton County and, if so, how and why?

U-Links projects have made far-reaching contributions to Haliburton County including municipal and countywide policy changes in regard to waste management, emergency preparedness, and active transportation; new programming and management plans for environmental education and monitoring; a ridesharing program in Haliburton county; hospice and palliative care services; and an integrative program focused on transitional support for youth with intellectual disabilities. The data suggest that relationships formed between organizations through community-based projects have had some of the greatest impacts on the Haliburton community regardless of immediate project outcomes.

Five factors that act as contexts, mechanisms, and outcomes of community-based research for U-Links are relevance, rigor, reach, relationships, and resources. These features can enable community-based research to contribute to community-wide social change, noting that generalizations beyond the U-Links case may not be possible. Although they are directly related to our case study of U-Links, future research will need to determine situations where these factors are relevant and generalizable.

Three of these factors—relevance, rigor, and reach—have been recognized as key factors influencing community-based research by Balazs and Morello-Frosch (2013). Our research suggests relationships and resources as additional factors, as they were important in creating a favorable context for projects to take place and to support the uptake of project results. We further suggest these five factors are both contextual and mechanistic factors that support community-based research processes, create benefits, and nurture social mobilization.

U-Links’ independence as a broker is important in creating relevant community impact and being responsive to community needs. U-Links has been able to overcome some of the traditional challenges of mobilizing community-based research in part because it is not a university entity. U-Links is a small community-based nonprofit with two or three employees and a volunteer Management Committee, of which 10 are community members and two represent university or faculty interests. This minimally bureaucratic organizational structure keeps U-Links grounded and accessible. U-Links exists first and foremost to satisfy the needs of the community rather than the university. The fact that projects are developed by organizations in this small, rural community through collaborative community-centered discussions and activities and then matched with Trent University is not only significant but also rather unique within community-campus partnerships (Walshe, 2017). This bottom-up approach keeps the projects rooted in Haliburton and ensures projects are matched to locally relevant needs.

The strength or rigor of the research itself is also important. Poorly completed projects simply do not produce results that can be directly implemented. Competent projects can be shared widely and can really make an impact. Students need to have the appropriate skills and faculty supports (which also fall under resources), and hosts need to take a role in mentorship to ensure this is achieved consistently. Furthermore, the scope of the project must be matched to the available time constraints and skills of students. Larger projects can be strategically broken down into smaller chunks for students and sequenced or marketed to graduate student researchers with appropriate capacity to address the project needs. Where projects fail, it is typically a combination of mismatched scoping, resources, and student skills and relationships within the research team that are the cause. U-Links must balance an inherent tension between wanting to match developed projects for the community and properly vetting student capabilities to ensure an effective match is made. The rigor of a project is a key mechanism for the research to be mobilized by project hosts and the broader community.

Reach—the publicity the research receives and who in the community (especially in terms of decision-makers) are engaged and aware of the work as it’s completed—is also important. Reach is a factor for establishing broader relationships and a mechanism for achieving sufficiently rigorous research outcomes. Reach is also a way to mobilize the research within the broader community. Projects gain traction by capitalizing on local concerns (relevance) and bringing people together through the collaborative process (relationships). When widely shared, community-based research projects have the capacity to inspire activism and further research on topics of concern.

The next factor is relationships. Preexisting relationships are the basis for much of the research U-Links develops. Relationships may also form between U-Links and faculty during the development of research and between multiple organizations to help support the research and co-host the students. In every case, the relationships that are formed between the host of a project, faculty members, students, and the broader community throughout the data collection process are key to ensuring a robust research outcome. The strength of these relationships, guided by values such as mutual respect and reciprocity, a shared passion and outlook, and clearly defined roles and responsibilities, enable the U-Links’ research to leave a social impact in the form of newfound coalitions and partnerships. Tensions and conflicts may arise within these partnerships, but U-Links’ role as a mediator to ensure equal voice to participants helps to manage these conflicts in a positive way, thus strengthening the bond and building trust. These relationships are a context for the development of future spin-off projects and a mechanism for strengthening capacity in the community for all future endeavors.

The resources made available through both academia and the community are both a limiting and enabling factor. Resources include access to funding, equipment, transportation, knowledge sources, social networks, and human resources. Typically, the university, host organization, and U-Links collaborate on creatively finding the resources required to develop, complete, or mobilize a project, with U-Links coordinating these efforts. Academic institutions have resources that organizations in Haliburton do not. Like many small nonprofits, U-Links often struggles to accommodate limited financial resources and internal capacity. While its independence helps U-Links’ work be relevant, it also limits their internal capacity and resources. U-Links does, however, have a wide social network within Haliburton and is able to lend social capital to projects when required. The availability of these resources (technical, social, and financial) are of utmost importance to the success of the project and must be considered in the design stage. By developing projects that respond to funding opportunities and ensuring resources are adequate before the project begins and throughout the research partnership, U-Links can create an environment in which students and hosts have what they need to be successful. Furthermore, applying for or earmarking funds for the implementation of research leads to a much greater chance of projects results being put into action. This factor is not always achieved directly through U-Links, but rather host organizations and the postsecondary institutions collaborate on budgets and resources to address these needs. Adequate resources are the context in which a project can be developed, a mechanism by which the project can be competently completed, and a key requirement for the project to be mobilized.

We illustrate the contexts, mechanisms, and outcomes for each stage of the community-based research project cycle in Figure 4. The project cycle shows how and when each factor transforms from a context, mechanism, or outcome, providing a list for other research brokering organizations to improve research outcomes for the community.

Figure 4.
Figure 4.
Figure 4.

Community-Based Research Project Cycle with Factors for Success.

6 Conclusion

The projects and partnerships enabled by U-Links Centre for Community-Based Research have led to capacity building and social change in Haliburton County by collaborating on relevant research, creating strong partnerships, supporting projects with adequate resources, and engaging the public in the dissemination of research findings. Regardless of the strength of individual community-based research projects, our evaluation suggests that the very act of bringing people together and engaging in community-based research can transform the social landscape, creating positive ripple effects for future community endeavors.

Acknowledgments

This research was co-designed by Emily Amon, Stephen Hill, and the U-Links Centre for Community-Based Research Management Committee, with financial support from the following sources: a research assistantship with the Community First: Impacts of Community Engagement (CFICE) project, a national seven-year Social Sciences and Humanities Research Council (SSHRC) research project fostering tools for community-based organizations and universities to collaborate and enact social change (Stack-Cutler, 2016), and a SSHRC Joseph Bombardier Graduate Scholarship.

References

Alcantara, L., Harper, G. W., & Keys, C. B. (2015). “There’s gotta be some give and take”: Community partner perspectives on benefits and contributions associated with community partnerships for youth. Youth and Society, 47(4), 462–485. doi:10.1177/0044118X12468141

Astbury, B., & Leeuw, F. L. (2010). Unpacking black boxes: Mechanisms and theory building in evaluation. American Journal of Evaluation, 31(3), 363–381. doi:10.1177/1098214010371972

Balazs, C. L., & Morello-Frosch, R. (2013). The three rs: How community-based participatory research strengthens the rigor, relevance, and reach of science. Environmental Justice, 6(1), 9–16. doi:10.1089/env.2012.0017

Befani, B. (2016). Choosing appropriate evaluation methods: A tool for assessment and selection. London: Bond. https://www.bond.org.uk/sites/default/files/caem_narrative_final_14oct16.pdf

Blamey, A., & Mackenzie, M. (2007). Theories of change and realistic evaluation: Peas in a pod or apples and oranges? Evaluation, 13(4), 439-455. doi:10.1177/1356389007082129

Bortolin, K. (2011). Serving ourselves: How the discourse on community engagement privileges the university over the community. Michigan Journal of Community Service Learning, 18(1), 49–58.

Bringle, R. G., & Hatcher, J. A. (2002). Campus-community partnerships: The terms of engagement. Journal of Social Issues, 58(3), 503–516.

Chevalier, J. M., & Buckles, D. J. (2013). Participatory action research: Theory and methods for engaged inquiry. Abingdon, UK: Routledge.

Cruz, N. I., & Giles, D. E., Jr. (2000). Where’s the community in service-learning research? Michigan Journal of Community Service Learning, (Special Issue), 28–34.

Funnell, S. C., & Rogers, P. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco, CA: Jossey-Bass.

Hawe, P., Shiell, A., & Riley, T. (2009). Theorising interventions as events in systems. American Journal of Community Psychology, 43(3–4), 267–276. doi:10.1007/s10464-009-9229-9

Jagosh, J., Bush, P. L., Salsberg, J., Macaulay, A. C., Greenhalgh, T., Wong, G., … Pluye, P. (2015). A realist evaluation of community-based participatory research: Partnership synergy, trust building and related ripple effects. BMC Public Health, 15(1). doi:10.1186/s12889-015-1949-1

Janzen, R., Ochocka, J., & Stobbe, A. (2017). Towards a theory of change for community-based research projects. Engaged Scholar Journal: Community-Engaged Research, Teaching, and Learning, 2(2), 44–64. doi:10.15402/esj.v2i2.165

Jonsen, K., & Jehn, K. A. (2009). Using triangulation to validate themes in qualitative studies. Qualitative Research in Organizations and Management: An International Journal, 4(2), 123–150. doi:10.1108/17465640910978391

Koleros, A., & Mayne, J. (2019). Using actor-based theories of change to conduct robust contribution analysis in complex settings. Canadian Journal of Program Evaluation, 33(3).

Leeuw, F. L. (2012). Linking theory-based evaluation and contribution analysis: Three problems and a few solutions. Evaluation, 18(3), 348–363. doi:10.1177/1356389012452051

Levkoe, C. Z., & Stack-Cutler, H. (2018). Brokering community-campus partnerships: An analytical framework. Gateways: International Journal of Community Research and Engagement, 11(1), 18–36. doi:10.5130/ijcre. v11i1.5527

Marullo, S., Cooke, D., Willis, J., Rollins, A., Burke, J., Bonilla, P., & Waldref, V. (2003). Community-based research assessments: Some principles and practices. Michigan Journal of Community Service Learning, 9(3), 57–68.

Mayne, J. (2008). Contribution analysis: An approach to exploring cause and effect (Brief No. 16). Rome, Italy: Institutional Learning and Change (ILAC).

Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270–280. doi:10.1177/1356389012451663Mayne, J. (2018). The COM.B ToC Model4 (No. 20). Working paper. Retrieved from https://www.researchgate.net/publication/323868561_The_COMB_ToC_Model4

Middlemiss, L. (2011). The power of community: How community-based organizations stimulate sustainable lifestyles among participants. Society & Natural Resources, 24(11), 1157–1173. doi:10.1080/08941920.2010.518582

Minkler, M., Vásquez, V. B., Tajik, M., & Petersen, D. (2008). Promoting environmental justice through community-based participatory research: The role of community and partnership capacity. Health Education & Behavior, 35(1), 119–137. doi:10.1177/1090198106287692

Morton, S. (2015). Progressing research impact assessment: A “contributions” approach. Research Evaluation, 24(4), 405–419. doi:10.1093/reseval/rvv016

Pawson, R., & Tilley, N. (1997). Realistic evaluation. London: Sage.

Reynolds, N. P. (2014). What counts as outcomes? Community perspectives of an engineering partnership. Michigan Journal of Community Service Learning, 21(1), 79–90.

Sandy, M., & Holland, B. (2006). Different worlds and common ground: Community partner perspectives on campus-community partnerships. Michigan Journal of Community Service Learning, 13(1), 30–43.

Savan, B. (2004). Community-university partnerships: Linking research and action for sustainable community development. Community Development Journal, 39(4), 372–384. doi:10.1093/cdj/bsh033

Statistics Canada. "2016 Community Profiles". Canada 2016 Census. February 21, 2017.

Stoecker, R. (2009). Are we talking the walk of community-based research? Action Research, 7(4), 385–404.

Strand, K., Marullo, S., Cutforth, N., Stoecker, R., & Donohue, P. (2003). Principles of best practice for community-based research. Michigan Journal of Community Service Learning, 9(3), 5–15.

Tennyson, R. (2005). The brokering guidebook: Navigating effective sustainable development partnerships. Retrieved from http://www.mspguide.org/sites/default/files/resource/brokering_guidebook_tenysson_2005.pdf

Valters, C. (2014). Theories of change in international development: Communication, learning, or accountability? (JSRP Paper 17). London: Justice Security and Research Programme, International Development Department, London School of Economics and Political Science.

Walshe, P. (2017). Trent University's community-based research partnership in Haliburton County. Trent Magazine Live. Retrieved from https://trentmagazine.ca/services-view/trent-universitys-community-based-research-partnership-haliburton-county/

Westhorp, G. (2014). Realist impact evaluation: An introduction. London: Overseas Development Institute. Retrieved from https://www.odi.org/publications/8716-realist-impact-evaluation-introduction

White, H., & Phillips, D. (2012). Addressing attribution of cause and effect in small n impact evaluations: Towards an integrated framework (Working Paper No. 15). New Delhi, India: International Initiative for Impact Evaluation.

Whynot, J., Lemire, S., & Montague, S. (2018). How we model matters: A manifesto for the next generation of program theorizing. Canadian Journal of Program Evaluation, 33(3). doi:10.3138/cjpe.53070

Authors

EMILY AMON is a recent graduate of the MA in Sustainability Studies at Trent University.

STEPHEN HILL is Associate Professor in the School of the Environment at Trent University.

JIM BLAKE is volunteer co-chair of the U-Links Centre for Community-Based Research Management Committee.

MARIE GAGE is volunteer member of the U-Links Centre for Community-Based Research Management Committee.