Quantitative

We analyzed item-level responses to the Civic-Minded Graduate Scale. Each survey item was a statement scored on a Likert scale between 1 (Strongly Disagree) and 9 (Strongly Agree). We calculated the mean and standard deviation of responses for each item and the percentage of students who disagreed with the statement (i.e., students who responded below a 5). A high percent disagreement or relatively low response when compared with other items on the construct may suggest that (1) the wording of the item is confusing, (2) the statement is not relevant to the respondents, or (3) the item is not relevant to the construct.

Qualitative

We sought qualitative evidence on whether the survey items were relevant to our participants. To do this, we performed content analysis on our interview data (Elo & Kyngäs, 2008). We identified interview responses in which students expressed sentiments related to survey items. For example, with the item “I am able to respond to others with empathy, regardless of their backgrounds,” we identified interview excerpts with empathy-related ideas such as listening or respecting others’ opinions or perspectives.

Quantitative

The quantitative phase of the concept-level analysis involved forming distinct factors to describe the underlying structure of civic-mindedness within our participant context. To identify factors, we performed principal and confirmatory factor analyses on multiple data sets. The results of the factor analyses showed five factors describing different aspects of civic-mindedness (see Appendix C). For a more detailed explanation of how we arrived at this factor solution, please refer to Hess et al. (2021). Here we report the factor loadings that we derived from the confirmatory factor analyses. Items with higher factor loadings are likely more salient to the factor within our participant context when compared with items with lower factor loadings (Fila et al., 2015).

Qualitative

While Hess et al. (2021) discussed how each of the five factors related to engineering education, this study also includes empirical evidence to identify how students perceived the relevance of each factor in engineering.

We analyzed the interviews using thematic analysis (Braun & Clarke, 2006; Terry et al., 2017) in NVivo. Rather than using the five factors as a deductive coding framework, we generated codes inductively and later compared alignment of emergent themes to the factors. We developed an initial coding scheme from analyzing three interviews and then refined the coding scheme with the remaining interviews. We met regularly through multiple stages of coding to discuss new insights and resolve disagreements until we finalized the coding scheme. We created codes for students’ perceptions of community engagement, their community involvement, and their beliefs about the role of engineers in communities.

We then compared codes and themes with the Civic-Minded Graduate Scale factors. We grouped together codes that were relevant to each factor and generated two to three themes related to each factor. These themes represent engineering students’ interpretations of each factor and how well the factors represent their experiences with civic engagement.

How did students quantitatively respond to individual items?

Students tended to respond affirmatively to the survey items associated with Factor 1, which suggests that students felt obligated to improve their communities through service. Students responded most positively to the item “I want to dedicate my career to improving society” (M = 6.98, SD = 1.70) and least positively to the item “I like to be involved in addressing community issues” (M = 6.58, SD = 1.63). The highest disagreement was 11.2% and in response to the item “I would say that the main purpose of work is to improve society through my career” (M = 6.67, SD = 1.76).

How and to what extent did interview responses align with survey item responses?

The four items associated with Factor 1 were evident in interviews, where students expressed that they enjoyed serving their community and felt a sense of responsibility towards their community. Some students felt a personal responsibility to improve society, mirroring the language found in the survey items, and a few students described how they aspired to use their engineering education to serve others. For example, Leo was drawn towards the opportunity to apply engineering to help others. He described his decision to enroll in a service-learning engineering course: “I was really interested in the community service aspect of [service-learning] and putting all of our young creative minds to work towards helping other people out.”

While Factor 1 items were not explicitly linked to the engineering profession, we inferred from interviews that some respondents were considering future engineering careers when responding to items relating to “my career” and “my work.” For example, Madison made her decision to major in engineering when she realized that by pursuing an engineering career, she could have a positive impact on other people. She described a conversation with her grandfather, an engineer, who encouraged her to study engineering. Her grandfather helped her become more cognizant of opportunities to engage with communities as an engineer, and Madison indicated that this conversation helped solidify her desire to pursue a career in engineering:

I told him, “I want to be an engineer, but I wouldn’t be helping the world or helping anyone. I would just be getting money, solving problems, doing my own thing. And I don’t know if that’s what I want to do.” He changed my perspective and said, “You’re going to have lots of knowledge and education and experience. You can use those abilities to help your community and help people in unique ways that will present themselves to you.” I found that interesting. I want to do something with my life that I love to do. But I also know that there will be ways where I can help people and solve problems.

In what other ways did interviews provide nuance to the survey items?

While Factor 1 items focused on improving one’s community, several students emphasized responsibility to avoid harming one’s community. For example, Amelia noted that at a minimum, people can contribute to their communities by simply not causing harm to others. As Amelia explained:

I think, as a general responsibility, just not doing harm. Not to say that you have to do good in the community. But you should definitely not do bad. You should help make the world better, but sometimes it’s all you can do to not make it worse.

How did students perceive this aspect of civic-mindedness with respect to the engineering profession?

We noted that students may have been considering the engineering profession when responding to survey items, but this was not explicitly prompted when we administered the survey. When asked about the responsibilities that engineers have towards communities, some students indicated that engineers had specialized skills that could help communities, such as problem-solving and generating solutions guided by their engineering thinking. Some students felt that engineers should use such expertise to improve their community. For example, Amelia said, “I think engineers have the responsibility to use what they’ve learned for the sake of helping the community, for specifically improving what they were taught to be experts on.”

While the survey responses suggested that many students sought to contribute to their own communities in their careers, this sentiment was seldom expressed during interviews. Nonetheless, most students felt that pursuing higher education in general gave them a responsibility to use their knowledge to help others. For example, Noah explained:

I feel like they [engineers] have a responsibility to use their education to give back to the community because not everybody goes to college and has this experience. Since they’re here, they should be able to give back to their community a little bit to help everyone.

How do the above findings align with engineering education literature?

Factor 1 addresses one’s personal and professional feelings of obligation to improve the lives of others. For engineers, these feelings of obligation can manifest as civic responsibilities to serve communities in a professional capacity. Civic responsibility connects engineers’ personal and professional roles within the communities where they live and work (Lin & Hess, 2021). Civic responsibility is one of several dimensions of responsibility in engineering ethics. Other dimensions include moral responsibility (Jing & Doorn, 2020), ethical responsibility (Herkert, 2005), professional responsibility (Lynch & Kline, 2000), and social responsibility (Canney & Bielefeldt, 2015). Like these other dimensions, civic responsibility emphasizes engineers’ obligations to public welfare, health, and safety (Lin & Hess, 2021).

Civic responsibility is also related to engineering ethics through a principlism framework (Beever & Brightman, 2016), which builds on deontological and utilitarian ethics. Deontology calls forth considerations of one’s duty to others, while utilitarianism seeks to maximize goodness. Factor 1 items cross these two thresholds by calling forth considerations of responsibility and doing good. Moreover, interview responses emphasize minimizing harm as a duty of engineers. These considerations show the alignment of civic considerations with ethics, which is a required learning outcome of ABET-accredited engineering programs (ABET, 2021).

How did students quantitatively respond to individual items?

Responses to the four survey items associated with Factor 2 tended to be positive, suggesting that students felt confident they could build consensus across differing viewpoints. Two survey items address self-efficacy beliefs in primarily interpersonal contexts and had positive responses. The other two items emphasized self-efficacy in civic contexts (i.e., helping solve social problems and having an impact on community problems). The Factor 2 item with the lowest mean response (M = 6.12) and the highest disagreement (17%) was “I am convinced that social problems are not too complex for me to help solve.”

How and to what extent did interview responses align with survey item responses?

While survey responses generally suggested that students were confident in responding to civic issues, in interviews, students seemed less confident in their ability to have difficult discussions and advance solutions to social problems. Moreover, among students who advocated for listening to and respecting others’ opinions on controversial social issues, some students preferred to avoid discussions on these topics, such as Larry who stated:

I really am not a politics guy. I don’t really like that scene. I think it’s mostly because it is a difficult conversation to have. And if I can avoid it, I don’t want to get anyone worked up over something. I feel it just usually leads to an argument and I’d like to avoid that if I can.

Several students expressed a lack of confidence in their ability to address community issues. For example, when asked to propose a solution to a community issue, Sara replied:

Talk to everyone you know and get them aware of these issues. Try to get as many people as you can to call local politicians. Go to city halls and let your local government know this is what you want changed. … I’m not entirely sure [if I would do this]. I’m a little scared of talking to random people. And I guess all of my friends are pretty like-minded in their political views. I’d have to really motivate myself to get out of my comfort zone, but I’d like to.

Though some students described actions they had taken in response to community issues through advocacy efforts, these examples tended to be pre-college. For example, Leo was part of a high school student group that leveraged the local newspaper and county council to increase access to public health in his community. Yet, like his peers, Leo found it difficult to continue similar engagement in college. Leo described his experience:

Back home, I was part of a few organizations and we always were able to get ahold of our county representations, the county council, the school board. The individual members on those boards, you could always email and get a response back in a few days, maybe a week later. I don’t think people in those types of positions here are quite as accessible.

In his hometown, Leo had access to social and political resources that helped him advance social causes in his community. However, in college he lacked these resources and, as a result, felt less confident in his ability to have a meaningful impact in his new community.

Given that “community” was not defined for participants in the survey, it is plausible that some participants were calling forth their pre-college experiences when responding to survey items, whereas others were possibly considering current course experiences or future career experiences that were more connected to engineering. Thus, there may be critical variation in the community that participants were considering, which we cannot ascertain based on our data collection approach. We recommend that future uses of the Civic-Minded Graduate Scale prime students to reflect on a specific community, be it pre-college, during college, or beyond college.

In what other ways did interviews provide nuance to the survey items?

The interviews showed the contextual nature of survey responses and variations in how students interpreted their communities therein. Moreover, the interviews revealed challenges students faced in their community involvement, including barriers that may inhibit student self-efficacy. For example, Sara proposed social organizing to respond to a community issue, but she did not feel prepared to undertake these actions herself. She expressed a desire to overcome her fear of sharing her views with strangers but was not confident that she could. Similarly, Isabella felt rallying community support was a feasible yet challenging way to address community needs. Like Sara, Isabella recognized the difficulties of creating social change:

Talking among the community about wanting to see a change in something that’s happening, potentially getting some kind of focus group started or petition to be signed—I feel like instigating change is a very difficult thing. So it probably comes at a really slow pace if things want to be accomplished.

How did students perceive this aspect of civic-mindedness with respect to the engineering profession?

When asked about ways that engineers could influence social issues, students tended to highlight engineers’ influence in the workplace rather than the collective influence engineers can have towards broader societal issues. For example, Madison said:

I don’t know how engineers specifically could get involved in social change, other than marches or standing up for what you believe in. Especially in the workplace, making sure everyone’s values are heard and who they are and what they look like and what they do.

In addition to focusing on the workplace, Madison provided examples of how engineers’ individual actions (rather than collective actions) could influence others, such as listening to others and standing up for their beliefs.

How do the above findings align with engineering education literature?

During interviews, students discussed social issues and enacting social change, but most examples of change efforts that students described tended to be pre-college and not specific to engineering. Moreover, participants’ exhibited a lack of confidence for sparking social change, which seemed aligned with a lack of awareness of how engineers, specifically, could bring their specialized knowledge to improve their communities. As first-year engineering students progress in their engineering programs and gain greater content knowledge and experiences, it is possible that they may also exhibit increased confidence in how to influence social issues.

Like Factor 1, students did not differentiate between duties of engineers versus people at large. Again, we connect these considerations to engineering ethics by considering the micro and macro-ethics distinction (Herkert, 2005). Micro-ethics involves considerations of individual responsibility and interpersonal relationships. In contrast, macro-ethics involves considerations of collective responsibility and broader impacts. The interview data suggests that students were primarily thinking of engineering social change in terms of micro-ethics, which suggests students may lack awareness of how engineers respond to macro-ethical issues. This aligns with the survey data; the lowest mean response was to the survey item “I am convinced that social problems are not too complex for me to help solve.” These findings support extant calls to engage engineering students in macro-ethical issues earlier in college (Herkert, 2005; Rottmann & Reeve, 2020).

How did students quantitatively respond to individual items?

Student responses to Factor 3 items were slightly positive, suggesting that students had some awareness of civic opportunities in college. Two survey items in Factor 3 related to students’ disciplinary knowledge and skills; these items had the highest agreement. The three remaining items showed relatively high levels of disagreement. Of particular note is the item that 38.7% of students disagreed with: “I would say that most other students know less about community organizations and volunteer opportunities than I do.” This item seems to prompt students to consider too many aspects simultaneously (e.g., other students, what other students know about community organizations, what other students know about volunteering opportunities, what “I” know about these things, and how what “I” know compares with what others know). Thus, we posit that many students struggled to interpret this question and that it ought to be simplified.

How and to what extent did interview responses align with survey item responses?

In both surveys and interviews, students recognized how engineering knowledge and skills could benefit community needs. We posit that many students were drawing on these considerations when responding to items. In addition, interviewees provided explicit examples of ways that they could apply knowledge from their courses to address community needs. For example, Julia, who was enrolled in a service-learning engineering course, explained:

This is my first experience of engineering and applying it to the community. Meeting the community partners that we work with and hearing them talk about their experience and why they need the product we’re working on, it showed me how the work I think I’m doing is contributing in the bigger picture.

Through service-learning, Julia learned how engineers could work alongside community partners to help address community needs. In addition, Isabella felt that she could apply her engineering learning to help her community:

These first-year engineering classes have been about general critical-thinking and programming and just really general concepts. Once I get into a job or a career, I’ll be doing exactly what they need, but I’ll be able to also retain some of the skills I’m not using. And potentially use them in a different area of volunteering, or some type of community involvement.

In what other ways did interviews provide nuance to the survey items?

Many students were knowledgeable about issues and opportunities within their communities. This was particularly evident among students who named specific community needs and potential solutions. However, few students were confident in their ability to address community issues. For example, Sara was not confident that she was well informed of issues on campus, but she was aware of the community’s need for an affordable grocery store:

I’m not very knowledgeable about problems at [this university]. I think one that I have noticed is food insecurity since there’s no grocery store very close to the campus. And at the cultural center I talked to, I brought up the [supermarket] that’s being built. They don’t seem to think that it’s going to be super affordable.

Since students were in their first year of college, their professional skills were still developing. Nonetheless, they recognized how engineers could use their skills within and outside of their professional work. Moreover, like Factor 2, these considerations suggest the importance of accounting for other variables that might affect student self-efficacy, such as having access to service opportunities, leveraging social networks, and using other forms of social capital.

How did students perceive this aspect of civic-mindedness with respect to the engineering profession?

Students recognized that engineers had specialized knowledge and skills that could be used to address community issues. For example, Leo, like many other students, recognized how engineers’ technical expertise and problem scoping skills could be applied to community improvement projects:

Engineers have very good technical problem-solving skills. They’re very analytical, they like to look at all the specifications of every single component that goes into the final project before they even decide to go forward in the project. Say a community wanted to build a new housing development: engineers could look at where the housing development is, what it’s going to impact, [and] they could put their analysis skills to use in providing feedback to leaders in their community.

How do the above findings align with engineering education literature?

Factor 3 represents one’s knowledge of community issues and how to address them. It connects to various efforts in engineering, particularly in humanitarian engineering (Hynes & Swenson, 2013), social justice (Riley, 2008), and engineering justice (Leydens & Lucena, 2018). One relevant example is the framework of engineering for and with people (Hynes & Swenson, 2013). While potentially a subtle distinction, engineering for people focuses on engineers as the specialized knower, whereas engineering with people suggests that engineers are individuals who bring their unique knowledge to community issues to collaborate in solving communal issues. Like Hynes and Swenson (2013), we question whether actual engagement with community members, potentially through service-learning contexts, might help students grapple with such considerations. Notably, this was a limited aspect of most of the community-oriented design projects that students engaged in during their first-year project-based design courses (see Appendix A).

How did students quantitatively respond to individual items?

Student responses to these items were mostly positive with few students disagreeing to the associated survey items, thus suggesting students felt confident they had the skills to navigate interpersonal relationships. The most positive response of all survey items was to the item “I am a good listener, even when peoples’ opinions are different from mine” (M = 7.52, SD = 1.44), followed closely by the item “I am able to respond to others with empathy, regardless of their backgrounds” (M = 7.46, SD = 1.37). Teaming, communication, listening, and empathy were a concerted focus of the first-year engineering program in which all students participated, and this instruction may have contributed to these positive responses.

How and to what extent did interview responses align with survey item responses?

In interviews, students raised the importance of listening to and respecting others’ viewpoints, though they did not mention empathy explicitly. Madison emphasized listening as a way to care for people in her community, stating: “I think a lot of people in the community need as much physical support with food and money as emotional support. So listening to your friends, active listening [is important].”

In addition to listening, students felt it was important to respect others’ opinions, especially when they differed from one’s own. Julia advised people to “speak out for whatever you believe in, and represent that idea in a good manner, not shaming people into joining you in your beliefs.” When conveying one’s opinions, Julia advocated for seeking to understand people who may disagree with you. Other students valued listening when discussing controversial issues, especially when those issues involved politics. For example, Larry talked about listening to people with differing political stances:

Everyone’s going to have a different opinion, so it is hard for people to get along. But I don’t think you have to get along, you just have to understand other people’s points of view and why they think that way.

Larry felt that having differing opinions should not preclude people from discussing and understanding others’ perspectives.

In what other ways did interviews provide nuance to the survey items?

In interviews, students emphasized the value of engaging with difference. Specifically, some students felt it was important to seek out and listen to diverse perspectives. Moreover, beyond listening to people with different viewpoints, Leo recognized the value of integrating diverse perspectives: “Having exposure to multiple different types of people, personalities, can help improve your own creative style and give you more ways to think or analyze different situations. … I actively seek to meet new people from many different backgrounds.”

Thus, Leo sought to engage people with different backgrounds and interests from him and valued their contributions to his thinking. He aspired to integrate diverse perspectives into his everyday life. Similarly, Julia appreciated how volunteering exposed her to people with different life experiences from her own:

I think that it’s important for everyone to do some volunteering work, because it does broaden your perspective and give you new insights by seeing what other people are going through. … [Volunteering] has exposed me to a lot of different people in the community. And getting to see their perspective and what they’re passionate about and their situation in life I think is really cool.

How did students perceive this aspect of civic-mindedness with respect to the engineering profession?

Interviewees recognized that collaboration and communication are important aspects of engineering. For example, Leo stated, “Engineers can solve important problems. And it doesn’t take too many people to solve a problem. It does take a team with lots of different viewpoints, but any sized team can achieve a goal in the end.” Leo explained that having a diverse team would help an engineering team work together more effectively.

While communication was an aspect of many responses, some students specifically recognized that listening was central to engineers’ job roles. John explained:

Engineers should listen to their clients. No matter which client I work with, I need to treat them well. It’s a service-based job, just like waiters, teachers; they all serve people. If you serve people, you should treat them well and do the right thing.

In addition to listening to one’s team members, John emphasized that engineers should listen to and value the opinions of people they serve.

How do the above findings align with engineering education literature?

Empathic interpersonal communication is important for engineers because they often work in teams and need to consider diverse viewpoints. Leydens and Lucena (2009) argued that listening was missing in engineering education, but they recognized that listening was necessary for effectively engaging community members. When engineers listen with empathy, they can integrate the perspectives and needs of community members into their work and empower community members with ownership over the project, which can improve the sustainability of the project (Leydens & Lucena, 2009). In addition, returning to the notion of engineering for and with community members (Hynes & Swenson, 2013), we argue that promoting civic-mindedness can involve teaching effective communication and collaboration skills to help students empathize with community members.

How did students quantitatively respond to individual items?

Average responses to all but one item were slightly positive, suggesting students intended to become engaged in civic and political issues. However, responses showed moderate disagreement compared with other factors, with more than 10% of respondents expressing disagreement to each item. The item with the highest disagreement, “I plan to participate in advocacy or political action groups after I graduate,” indicated that 40% of students were not planning to join advocacy or political action groups after graduation. This high disagreement could suggest these first-year engineering students are not yet interested in using political engagement to enact social change, or it may suggest that advocacy and political action groups are not salient to students’ future careers. Thus, based on these responses, when used in the context of engineering, these items may need to be rephrased or broadened to be more relevant to the engineering profession.

How and to what extent did interview responses align with survey item responses?

In interviews, students agreed it was important to stay informed and engage in political processes, though students varied in their perceptions regarding what level of involvement engineers should have. Some students became more interested in politics in college than in high school. For example, Sara became increasingly aware of social inequities and viewed politics as a path for social change:

I’m definitely more interested in politics now than I was as a high schooler. Just becoming more aware of the problems in America, everyone has someone in their life most likely that is affected by some pitfall in American society.

Conversely, other students noted that they were less involved with the news and politics in college. Isabella indicated she lacked the time to keep up with the news: “I find going online to search up things and find the correct news instead of the fake news, it just takes so much time out of my day. I don’t have that time anymore; I’m spending it studying.” Though Isabella wanted to follow current events, she found it difficult to devote her time to finding credible news sources. This challenge was common among the interviewed students.

In what other ways did interviews provide nuance to the survey items?

Though students recognized the importance of political engagement, they acknowledged barriers that made it difficult to become politically involved. Many students were deterred by political conflicts and sought to avoid the emotionally charged nature of political discussions. For example, Grace noted the mental toll of participating in politics:

[Politics is] emotionally exhausting. Honestly, I don’t know if I always have the emotional capacity to deal with it. … If it’s something that you genuinely feel is an important cause, you should have involvement in it. But if you cannot commit the mental faculties to being involved, then I feel like it’s not as much pressure as community involvement. Because it’s usually a larger scale.

Grace preferred to reserve her energy for forms of community engagement other than politics that were less emotionally taxing and resulted in a more tangible impact on her community. Like Grace, many students valued the impact of politics and wanted to be politically involved, though some students acknowledged they did not always follow through on their intentions to stay updated on contemporary news and politics issues.

The misalignment between perceived political importance and engagement was mirrored in the survey results, where students responded more affirmatively about their intent to stay current with news rather than their keeping up with local community issues. This discrepancy may be explained by students choosing to follow national news more closely than local news. Most students did not discuss local politics whatsoever. Even when students brought up issues in their local communities, they mostly discussed issues facing the campus community rather than the surrounding community outside the university.

How did students perceive this aspect of civic-mindedness with respect to the engineering profession?

We analyzed students’ beliefs about the relationality between engineering and politics and found a range of responses. Students readily described ways that politics influenced engineers, though they struggled to identify ways that engineers could influence politics. When asked about how politics and engineering relate, Leo explained:

They’re so very different. Unless you’re an engineer in politics, I’m not sure how you would influence politics that much, besides casting your vote and just talking with the people around you. … Politicians control funding for organizations that do a lot of engineering and science. … Politicians can use their control of money to focus engineering towards different sectors that they value.

Leo acknowledged the influence of politics on engineering, namely politicians prioritizing certain industry sectors with funding, though he was unsure about ways that engineers could engage in politics. Yet many other students recognized how engineers and politics could work together to address social issues. For example, Sara described how engineers and the government could collectively address climate change:

I think engineering right now should focus on the environment. Things need to change at a government level with regulations. … I think cleaning the environment and preventing more damage is where engineers need to focus more. I think making engineers to make these innovations, and getting the government to become stricter, go hand and hand.

Sara expressed her opinion that governments needed to impose stricter regulations and engineers needed to innovate for more sustainable technologies. Her example of climate change shows how societal challenges can benefit from engineers and politics working in tandem.

How do the above findings align with engineering education literature?

Factor 5 captures one’s intentions to be politically engaged inside or outside of engineering. Our findings suggest that engineering students faced barriers to their political involvement and felt politics and engineering were disconnected. This disconnect reflects a culture of depoliticization in engineering that demarcates engineering as a purely “technical” space detached from political considerations (Cech & Sherick, 2015). Students’ socialization into a culture of depoliticization, coupled with their discomfort with political discussions, suggest a concerning trend of students shying away from political engagement in their personal and professional lives. Morgan et al. (2020) identified challenges that engineering students face in their political involvement, including time constraints from academics and experiences that relegated politics to the periphery of engineering. Like Morgan et al. (2020), we recommend engineering instructors identify ways to help students engage meaningfully with politics within curriculum. By addressing politics within engineering curriculum, students can see examples of the connections between engineering and politics.

1. When administering survey instruments about civic outcomes, prime students to respond with a specific context in mind.

When we administered the Civic-Minded Graduate Scale, we did not give students a particular frame or context to consider as they responded to the survey items. Thus, while some items prompted students to reflect on community, we cannot be certain how survey respondents defined community when responding to these items. However, in interviews, we noticed variations in how students scoped their community. Many students identified with multiple communities, including their hometowns, residence halls, the campus community, and the local community. We recommend asking students to consider a specific community context when responding to the survey, especially on items pertaining to civic attitudes and behaviors. Priming responses to focus on a specific community keeps students’ responses consistent and is thus potentially more useful and valid for comparison tests and other analyses.

Context is also important to consider because civic outcomes can manifest differently based on one’s environment. For example, prior studies have shown that civic engagement decreases as students graduate from high school and transition into early adulthood (Finlay et al., 2010; Malin et al., 2017). These changes in civic participation have been attributed in part to changes in students’ community context, such as the loss of supportive structures (i.e., family and social groups) and increasing barriers (i.e., academic demands on time) (Finlay et al., 2010; Malin et al., 2017). By focusing on a specific context, we can understand factors in a particular environment that promote or inhibit civic involvement.

2. Consider utilizing mixed methods to assess civic learning outcomes.

In this study, we outlined a mixed methods approach to validating the Civic-Minded Graduate Scale. This approach can be applied to other instruments in a variety of contexts at the course, department, or university levels. We reflect on our methods to offer some insights into using mixed methods for assessing civic outcomes. This study began with factor analyses to generate five constructs explaining the underlying structure of civic-mindedness as measured by the Civic-Minded Graduate Scale. We then conducted interviews to gather qualitative evidence on how well these five constructs accounted for civic-mindedness within the context of first-year engineering students.

The interviews revealed some limitations of relying on survey data alone. While the quantitative data helped us understand trends in students’ attitudes and beliefs, the qualitative data revealed important nuances in students’ interpretations of civic-mindedness. Furthermore, the Civic-Minded Graduate Scale focuses largely on students’ intentions, attitudes, and beliefs towards civic engagement rather than their behavior. The qualitative data captured environmental factors that explained why civic-minded dispositions did not necessarily translate into civic engagement. Though many students believed it was important to be involved in their communities, they were generally not as involved as they hoped to be. The most common barrier to involvement was the espoused pressure to prioritize academics over service. Interviewees who remained most civically engaged integrated service with academics through service-learning or social groups. Though we acknowledge mixed methods research adds time and resources to a study, these findings indicate how meaningfully engaging distinct data sources can lead to significant insights (Greene, 2007). Indeed, the Civic-Minded Graduate Scale was initially developed and validated using multiple forms of quantitative and qualitative evidence. Thus, we encourage researchers interested in civic-mindedness to collect qualitative data alongside use of the Civic-Minded Graduate Scale in future research.