Higher education aims to prepare students for civic responsibility and engaged citizenship by instilling civic values and promoting meaningful civic engagement (Ehrlich, 2000). Along with pedagogies to enhance civic learning, educators need valid methods to assess whether these efforts lead to desired civic learning outcomes. While civic learning is aligned with many facets of engineering curricula, such as service-learning, ethics, and empathy, civics is rarely an explicit focus in engineering education literature. As a result, while many tools to evaluate civic learning have been developed outside of engineering, few tools have been tested within the specific context of engineering. This study aims to address that gap by validating one instrument—the Civic-Minded Graduate Scale—in the context of first-year engineering students.
The Civic-Minded Graduate Scale (Steinberg et al., 2011) provides a quantitative assessment of civic learning outcomes that represent the construct of civic-mindedness. This instrument can serve as a guide for assessing undergraduate students’ “capacity and desire to work with others to achieve the common good” (Steinberg et al., 2011, p. 20). While the instrument is theoretically applicable across disciplines, this study focuses on the instrument’s validity in the context of first-year engineering students. In addition, we consider this study’s findings to align with the results and approaches of prior validation studies (Bringle et al., 2019; Bringle & Wall, 2020; Hess et al., 2021).
This study aims to understand and assess first-year engineering students’ attitudes towards civic-mindedness. Specifically, two research questions are addressed:
To what extent do the Civic-Minded Graduate Scale constructs represent first-year engineering students’ civic-mindedness?
In what ways do the Civic-Minded Graduate Scale and its underlying constructs manifest in first-year engineering students’ perceptions about the engineering profession?
This study integrates quantitative and qualitative methods to contribute multiple forms of evidence for validity (Fila et al., 2015) within the context of first-year engineering students. Although these research questions focus on engineering, this study’s findings can inform research and practice outside of engineering, as described in the implications section. While focusing on first-year students limits generalizability to higher academic standings, the first year of college is a particularly critical time to study as most students are learning and internalizing the norms of their future profession for the first time.
Background
Why is civic-mindedness important for engineers?
As professionals, engineers have a responsibility to serve public welfare and community interests (Hynes & Swenson, 2013). Engineers can contribute their professional knowledge and skills to address community needs and improve the livelihoods of community members (Peters, 2004; Tsang, 2000). To do so, engineers need to develop skills to effectively listen to community partners’ needs and be able to develop meaningful relationships with community partners (Schneider et al., 2008). Thus, engineering education needs pedagogical approaches that help students develop skills to work with communities effectively (Tsang, 2000). To understand the impact of such pedagogical approaches, educators need valid methods to assess whether these efforts lead to desired civic learning outcomes.
How can we assess civic-mindedness?
Researchers have developed instruments to assess civic outcomes, including civic knowledge, skills, attitudes, and behaviors. While civic knowledge and skills can be observed or measured directly, civic attitudes often rely on self-reported measures (Reason & Hemer, 2015). Given that civic attitudes are quite broad in scope, these instruments often include subscales that highlight different features of civic attitudes or dispositions (Torney-Purta et al., 2015).
The Civic-Minded Graduate Scale was designed with four conceptual domains—civic knowledge, skills, dispositions, and behavioral intentions. These domains contain one to three subdomains. In practice, the Civic-Minded Graduate Scale has been applied as a unidimensional construct for civic-mindedness, with a single factor accounting for 45% to 50% of the variance in the phenomenon of civic-mindedness (Steinberg et al., 2011). Additional studies have evaluated the Civic-Minded Graduate Scale’s construct and convergent validity by comparing its responses with those of related scales (Bringle et al., 2019; Bringle & Wall, 2020).
How can we appropriately use the Civic-Minded Graduate Scale in engineering education?
Hess et al. (2021) tested the structural validity of the Civic-Minded Graduate Scale with science and engineering students. Hess et al. employed principal component analyses to identify a potential five-factor structure for the Civic-Minded Graduate Scale and then confirmed its structural robustness through confirmatory factor analyses. As opposed to the four domains of the original Civic-Minded Graduate Scale, the resulting factors included: (1) valuing community engagement, (2) confidence in influencing social issues, (3) civic knowledge and skills, (4) empathic interpersonal communication, and (5) civic intentions and obligations. This five-factor solution explained 57.5% of the variance and retained 21 of the 30 original survey items.
This study builds on Hess et al.’s (2021) recent factor analysis of the Civic-Minded Graduate Scale with first-year engineering students by bringing qualitative analyses to the five-factor structure.
Why is the first year of college important to study?
This study focuses on first-year students because the first year of college represents a critical time when students learn the values and norms of their institution (Manning-Ouellette & Hemer, 2019; Mayhew & Engberg, 2011) and profession (Dall’Alba, 2009). Thus, the first year of college is a particularly ripe time to engage students in civic engagement (Manning-Ouellette & Hemer, 2019; Mayhew & Engberg, 2011) and foster their sense of civic responsibility (Hunter & Moody, 2009). Towards these aims, educators have increasingly adopted service-learning pedagogies in first-year curriculum (Bauer et al., 2015).
Within engineering programs, service-learning often occurs in first-year engineering project courses (Bielefeldt & Canney, 2014). Many first-year engineering courses also incorporate project-based learning, which presents natural opportunities for service-learning (Scherrer & Sharpe, 2020). In addition, project-based and service-learning courses have been linked to several positive outcomes for first-year engineering students, including developing professional skills (Shelby et al., 2013), fostering prosocial attitudes towards social responsibility (Bielefeldt & Canney, 2014), and increasing retention (Dym et al., 2005). Thus, first-year engineering curricula can play a significant role in shaping engineering students’ civic-mindedness, especially when project-based and service-learning pedagogies are introduced.
Methods
This study uses a mixed methods approach by triangulating quantitative and qualitative evidence to validate the Civic-Minded Graduate Scale among first-year engineering students.
Data Collection
The context for this study was a large, public university in the Midwest. We recruited participants from an engineering course required for first-year engineering majors. Though the course was not focused on service-learning, the course included community-oriented design projects. We provide more details associated with the design project, course learning objectives, and community-oriented aspects of the design project in Appendix A. We administered the Civic-Minded Graduate Scale (Appendix B) at the end of the course and achieved a response rate of 73% (n = 419). Table 1 provides demographic statistics, which were largely aligned with the university’s undergraduate engineering student population.
Demographic Information of Survey Respondents (n = 419)
| Demographic item | Number |
|---|---|
| Gender | |
| Male | 312 (74%) |
| Female | 104 (25%) |
| Not declared | 3 (1%) |
| Race/ethnicity | |
| American Indian or Alaska Native | 0 |
| Asian | 107 (25.5%) |
| Black or African American | 6 (1%) |
| Hispanic or Latino | 17 (4%) |
| Native Hawaiian or Pacific Islander | 0 |
| White or Caucasian | 250 (60%) |
| Other race/ethnicity | 2 (0.5%) |
| Multi-racial | 29 (7%) |
| Not declared | 8 (2%) |
| Age (M, SD) | 18, 0.60 |
Following the survey, we recruited participants from the same population for interviews. We interviewed 11 first-year engineering students, six of whom were women and five were men. The interviews were semi-structured, beginning with questions about students’ experiences with community involvement before and during college. We adapted the interview protocol developed by Steinberg et al. (2011) and asked students about their experiences with community service, political involvement, and social change.
Data Analyses
We analyzed the quantitative and qualitative data separately. We then integrated the data at the item and concept levels to interpret results. At the item level, we analyzed individual survey items and checked the salience of each survey item within the interviews. At the concept level, we focused on how the survey items were grouped into constructs. Table 2 summarizes our strategy for mixing the data in our analyses.
Data Analysis and Mixing Strategies (adapted from Fila et al., 2015)
| Level | Guiding questions | Quantitative | Qualitative |
|---|---|---|---|
| Item | Are the survey items representative of the survey concepts and are they accurately interpreted in the context? | Item analysis | Content analysis |
| Concept | Is the construct relevant in the participant context? | Factor analysis | Thematic analysis |
Item level
Quantitative
We analyzed item-level responses to the Civic-Minded Graduate Scale. Each survey item was a statement scored on a Likert scale between 1 (Strongly Disagree) and 9 (Strongly Agree). We calculated the mean and standard deviation of responses for each item and the percentage of students who disagreed with the statement (i.e., students who responded below a 5). A high percent disagreement or relatively low response when compared with other items on the construct may suggest that (1) the wording of the item is confusing, (2) the statement is not relevant to the respondents, or (3) the item is not relevant to the construct.
Qualitative
We sought qualitative evidence on whether the survey items were relevant to our participants. To do this, we performed content analysis on our interview data (Elo & Kyngäs, 2008). We identified interview responses in which students expressed sentiments related to survey items. For example, with the item “I am able to respond to others with empathy, regardless of their backgrounds,” we identified interview excerpts with empathy-related ideas such as listening or respecting others’ opinions or perspectives.
Concept level
Quantitative
The quantitative phase of the concept-level analysis involved forming distinct factors to describe the underlying structure of civic-mindedness within our participant context. To identify factors, we performed principal and confirmatory factor analyses on multiple data sets. The results of the factor analyses showed five factors describing different aspects of civic-mindedness (see Appendix C). For a more detailed explanation of how we arrived at this factor solution, please refer to Hess et al. (2021). Here we report the factor loadings that we derived from the confirmatory factor analyses. Items with higher factor loadings are likely more salient to the factor within our participant context when compared with items with lower factor loadings (Fila et al., 2015).
Qualitative
While Hess et al. (2021) discussed how each of the five factors related to engineering education, this study also includes empirical evidence to identify how students perceived the relevance of each factor in engineering.
We analyzed the interviews using thematic analysis (Braun & Clarke, 2006; Terry et al., 2017) in NVivo. Rather than using the five factors as a deductive coding framework, we generated codes inductively and later compared alignment of emergent themes to the factors. We developed an initial coding scheme from analyzing three interviews and then refined the coding scheme with the remaining interviews. We met regularly through multiple stages of coding to discuss new insights and resolve disagreements until we finalized the coding scheme. We created codes for students’ perceptions of community engagement, their community involvement, and their beliefs about the role of engineers in communities.
We then compared codes and themes with the Civic-Minded Graduate Scale factors. We grouped together codes that were relevant to each factor and generated two to three themes related to each factor. These themes represent engineering students’ interpretations of each factor and how well the factors represent their experiences with civic engagement.
Validity
A mixed methods approach aims to integrate different processes of knowledge generation, or what Greene (2007) describes as “meaningful engagement with difference” (p. 14). To facilitate such engagement, in alignment with Fila et al. (2015), we employed Messick’s (1995) six aspects of construct validity to bolster our confidence in our findings. Table 3 provides an overview of these validity aspects and the associated strategies used in this study.
Overview of Validity Types and Strategies (adapted from Messick, 1995)
| Validity type | Goal | Strategy |
|---|---|---|
| Content | Ensure content aligns with target phenomena | Employ content and item analysis to provide evidence of content relevance |
| Substantive | Provide theoretical rationales for observations | Identify how participant experiences align with the Civic-Minded Graduate framework |
| Structural | Identify the “extent to which the internal structure of the assessment” represents the structure of the phenomena | Conduct Principal Factor Analysis to identify a potential factor structure followed by Confirmatory Factor Analysis to confirm that structure with two subsets of students |
| Generalizability | Consider if and how findings translate to other contexts | Triangulate findings with prior literature, within and beyond engineering |
| External | Identify convergent and divergent evidence | Employ a multi-methods approach and compare consistencies and inconsistencies across responses |
| Consequential | Identify how the results prompt change | In the implications section of this manuscript, identify how these findings can be utilized by instructors and programs |
Results & Discussion
We present quantitative and qualitative evidence for the five-factor structure of the Civic-Minded Graduate Scale within the context of first-year engineering students. We organize our findings around the five factors. For each factor, we address these sub-research questions:
How did students quantitatively respond to individual items associated with the factor, including mean response, variation, and percent disagreement?
How and to what extent did interview responses align with survey item responses?
In what other ways did interviews provide nuance to the survey items?
How did students perceive this aspect of civic-mindedness with respect to the engineering profession?
How do the above findings align with engineering education literature?
Factor 1: Valuing community engagement
Factor 1 includes four items and addresses students’ feelings of obligation to contribute to society and improve the lives of others (see Table 4).
Factor 1 Survey Items and Analyses
| Survey item | Item level | Concept level | ||
|---|---|---|---|---|
| Mean ± SD (% disagree) (n = 419) | Content analysis | Factor loadings | Themes | |
| I want to dedicate my career to improving society. | 6.98 ± 1.70 (8.1%) | Some students mentioned wanting to help others in their careers. | 0.86 | Students enjoyed serving others. |
| I believe that I have a responsibility to use the knowledge that I have gained to serve others. | 6.84 ± 1.65 (8.4%) | Some students felt people with higher education should use their knowledge to serve others. | 0.78 | Students sought ways to contribute to their communities. |
| I would say that the main purpose of work is to improve society through my career. | 6.67 ± 1.76 (11.2%) | Students did not describe improving society as the primary purpose of their work. | 0.86 | Students believed their responsibilities included improving their community as well as avoiding causing any harm to communities. |
| I like to be involved in addressing community issues. | 6.58 ± 1.63 (9.8%) | Many students said they enjoyed community service. | 0.79 | Some students also wanted to contribute to their communities in their professional lives. |
-
Note. Responses were on a 9-point Likert-type scale where 1 = Strongly Disagree and 9 = Strongly Agree.
How did students quantitatively respond to individual items?
Students tended to respond affirmatively to the survey items associated with Factor 1, which suggests that students felt obligated to improve their communities through service. Students responded most positively to the item “I want to dedicate my career to improving society” (M = 6.98, SD = 1.70) and least positively to the item “I like to be involved in addressing community issues” (M = 6.58, SD = 1.63). The highest disagreement was 11.2% and in response to the item “I would say that the main purpose of work is to improve society through my career” (M = 6.67, SD = 1.76).
How and to what extent did interview responses align with survey item responses?
The four items associated with Factor 1 were evident in interviews, where students expressed that they enjoyed serving their community and felt a sense of responsibility towards their community. Some students felt a personal responsibility to improve society, mirroring the language found in the survey items, and a few students described how they aspired to use their engineering education to serve others. For example, Leo was drawn towards the opportunity to apply engineering to help others. He described his decision to enroll in a service-learning engineering course: “I was really interested in the community service aspect of [service-learning] and putting all of our young creative minds to work towards helping other people out.”
While Factor 1 items were not explicitly linked to the engineering profession, we inferred from interviews that some respondents were considering future engineering careers when responding to items relating to “my career” and “my work.” For example, Madison made her decision to major in engineering when she realized that by pursuing an engineering career, she could have a positive impact on other people. She described a conversation with her grandfather, an engineer, who encouraged her to study engineering. Her grandfather helped her become more cognizant of opportunities to engage with communities as an engineer, and Madison indicated that this conversation helped solidify her desire to pursue a career in engineering:
I told him, “I want to be an engineer, but I wouldn’t be helping the world or helping anyone. I would just be getting money, solving problems, doing my own thing. And I don’t know if that’s what I want to do.” He changed my perspective and said, “You’re going to have lots of knowledge and education and experience. You can use those abilities to help your community and help people in unique ways that will present themselves to you.” I found that interesting. I want to do something with my life that I love to do. But I also know that there will be ways where I can help people and solve problems.
In what other ways did interviews provide nuance to the survey items?
While Factor 1 items focused on improving one’s community, several students emphasized responsibility to avoid harming one’s community. For example, Amelia noted that at a minimum, people can contribute to their communities by simply not causing harm to others. As Amelia explained:
I think, as a general responsibility, just not doing harm. Not to say that you have to do good in the community. But you should definitely not do bad. You should help make the world better, but sometimes it’s all you can do to not make it worse.
How did students perceive this aspect of civic-mindedness with respect to the engineering profession?
We noted that students may have been considering the engineering profession when responding to survey items, but this was not explicitly prompted when we administered the survey. When asked about the responsibilities that engineers have towards communities, some students indicated that engineers had specialized skills that could help communities, such as problem-solving and generating solutions guided by their engineering thinking. Some students felt that engineers should use such expertise to improve their community. For example, Amelia said, “I think engineers have the responsibility to use what they’ve learned for the sake of helping the community, for specifically improving what they were taught to be experts on.”
While the survey responses suggested that many students sought to contribute to their own communities in their careers, this sentiment was seldom expressed during interviews. Nonetheless, most students felt that pursuing higher education in general gave them a responsibility to use their knowledge to help others. For example, Noah explained:
I feel like they [engineers] have a responsibility to use their education to give back to the community because not everybody goes to college and has this experience. Since they’re here, they should be able to give back to their community a little bit to help everyone.
How do the above findings align with engineering education literature?
Factor 1 addresses one’s personal and professional feelings of obligation to improve the lives of others. For engineers, these feelings of obligation can manifest as civic responsibilities to serve communities in a professional capacity. Civic responsibility connects engineers’ personal and professional roles within the communities where they live and work (Lin & Hess, 2021). Civic responsibility is one of several dimensions of responsibility in engineering ethics. Other dimensions include moral responsibility (Jing & Doorn, 2020), ethical responsibility (Herkert, 2005), professional responsibility (Lynch & Kline, 2000), and social responsibility (Canney & Bielefeldt, 2015). Like these other dimensions, civic responsibility emphasizes engineers’ obligations to public welfare, health, and safety (Lin & Hess, 2021).
Civic responsibility is also related to engineering ethics through a principlism framework (Beever & Brightman, 2016), which builds on deontological and utilitarian ethics. Deontology calls forth considerations of one’s duty to others, while utilitarianism seeks to maximize goodness. Factor 1 items cross these two thresholds by calling forth considerations of responsibility and doing good. Moreover, interview responses emphasize minimizing harm as a duty of engineers. These considerations show the alignment of civic considerations with ethics, which is a required learning outcome of ABET-accredited engineering programs (ABET, 2021).
Factor 2: Confidence in influencing social issues
Factor 2 includes four items and encompasses students’ self-efficacy in influencing social issues (see Table 5).
Factor 2 Survey Items and Analyses
| Survey item | Item level | Concept level | ||
|---|---|---|---|---|
| Mean ± SD (% disagree) (n = 419) | Content analysis | Factor loadings | Themes | |
| Other students who know me well would describe me as a person who can discuss controversial social issues with civility and respect. | 6.80 ± 1.70 (8.8%) | Students advocated for listening to and respecting others’ opinions on controversial social issues. | 0.76 | Some students described ways they took action in response to community issues through advocacy. Some students felt a lack of agency to make a meaningful impact in their local community. Some students preferred to avoid contentious topics, especially political discussions. |
| I believe that having an impact on community problems is within my reach. | 6.76 ± 1.50 (6.4%) | Students recognized ways they could impact community problems. | 0.68 | |
| I have often been able to persuade others to agree with my point of view. | 6.72 ± 1.49 (6.7%) | Some students acknowledged the difficulties of persuading others. | 0.50 | |
| I am convinced that social problems are not too complex for me to help solve. | 6.12 ± 1.75 (17.0%) | On average, students tended to not feel confident in their ability to influence social change. | 0.56 | |
-
Note. Responses were on a 9-point Likert-type scale where 1 = Strongly Disagree and 9 = Strongly Agree.
How did students quantitatively respond to individual items?
Responses to the four survey items associated with Factor 2 tended to be positive, suggesting that students felt confident they could build consensus across differing viewpoints. Two survey items address self-efficacy beliefs in primarily interpersonal contexts and had positive responses. The other two items emphasized self-efficacy in civic contexts (i.e., helping solve social problems and having an impact on community problems). The Factor 2 item with the lowest mean response (M = 6.12) and the highest disagreement (17%) was “I am convinced that social problems are not too complex for me to help solve.”
How and to what extent did interview responses align with survey item responses?
While survey responses generally suggested that students were confident in responding to civic issues, in interviews, students seemed less confident in their ability to have difficult discussions and advance solutions to social problems. Moreover, among students who advocated for listening to and respecting others’ opinions on controversial social issues, some students preferred to avoid discussions on these topics, such as Larry who stated:
I really am not a politics guy. I don’t really like that scene. I think it’s mostly because it is a difficult conversation to have. And if I can avoid it, I don’t want to get anyone worked up over something. I feel it just usually leads to an argument and I’d like to avoid that if I can.
Several students expressed a lack of confidence in their ability to address community issues. For example, when asked to propose a solution to a community issue, Sara replied:
Talk to everyone you know and get them aware of these issues. Try to get as many people as you can to call local politicians. Go to city halls and let your local government know this is what you want changed. … I’m not entirely sure [if I would do this]. I’m a little scared of talking to random people. And I guess all of my friends are pretty like-minded in their political views. I’d have to really motivate myself to get out of my comfort zone, but I’d like to.
Though some students described actions they had taken in response to community issues through advocacy efforts, these examples tended to be pre-college. For example, Leo was part of a high school student group that leveraged the local newspaper and county council to increase access to public health in his community. Yet, like his peers, Leo found it difficult to continue similar engagement in college. Leo described his experience:
Back home, I was part of a few organizations and we always were able to get ahold of our county representations, the county council, the school board. The individual members on those boards, you could always email and get a response back in a few days, maybe a week later. I don’t think people in those types of positions here are quite as accessible.
In his hometown, Leo had access to social and political resources that helped him advance social causes in his community. However, in college he lacked these resources and, as a result, felt less confident in his ability to have a meaningful impact in his new community.
Given that “community” was not defined for participants in the survey, it is plausible that some participants were calling forth their pre-college experiences when responding to survey items, whereas others were possibly considering current course experiences or future career experiences that were more connected to engineering. Thus, there may be critical variation in the community that participants were considering, which we cannot ascertain based on our data collection approach. We recommend that future uses of the Civic-Minded Graduate Scale prime students to reflect on a specific community, be it pre-college, during college, or beyond college.
In what other ways did interviews provide nuance to the survey items?
The interviews showed the contextual nature of survey responses and variations in how students interpreted their communities therein. Moreover, the interviews revealed challenges students faced in their community involvement, including barriers that may inhibit student self-efficacy. For example, Sara proposed social organizing to respond to a community issue, but she did not feel prepared to undertake these actions herself. She expressed a desire to overcome her fear of sharing her views with strangers but was not confident that she could. Similarly, Isabella felt rallying community support was a feasible yet challenging way to address community needs. Like Sara, Isabella recognized the difficulties of creating social change:
Talking among the community about wanting to see a change in something that’s happening, potentially getting some kind of focus group started or petition to be signed—I feel like instigating change is a very difficult thing. So it probably comes at a really slow pace if things want to be accomplished.
How did students perceive this aspect of civic-mindedness with respect to the engineering profession?
When asked about ways that engineers could influence social issues, students tended to highlight engineers’ influence in the workplace rather than the collective influence engineers can have towards broader societal issues. For example, Madison said:
I don’t know how engineers specifically could get involved in social change, other than marches or standing up for what you believe in. Especially in the workplace, making sure everyone’s values are heard and who they are and what they look like and what they do.
In addition to focusing on the workplace, Madison provided examples of how engineers’ individual actions (rather than collective actions) could influence others, such as listening to others and standing up for their beliefs.
How do the above findings align with engineering education literature?
During interviews, students discussed social issues and enacting social change, but most examples of change efforts that students described tended to be pre-college and not specific to engineering. Moreover, participants’ exhibited a lack of confidence for sparking social change, which seemed aligned with a lack of awareness of how engineers, specifically, could bring their specialized knowledge to improve their communities. As first-year engineering students progress in their engineering programs and gain greater content knowledge and experiences, it is possible that they may also exhibit increased confidence in how to influence social issues.
Like Factor 1, students did not differentiate between duties of engineers versus people at large. Again, we connect these considerations to engineering ethics by considering the micro and macro-ethics distinction (Herkert, 2005). Micro-ethics involves considerations of individual responsibility and interpersonal relationships. In contrast, macro-ethics involves considerations of collective responsibility and broader impacts. The interview data suggests that students were primarily thinking of engineering social change in terms of micro-ethics, which suggests students may lack awareness of how engineers respond to macro-ethical issues. This aligns with the survey data; the lowest mean response was to the survey item “I am convinced that social problems are not too complex for me to help solve.” These findings support extant calls to engage engineering students in macro-ethical issues earlier in college (Herkert, 2005; Rottmann & Reeve, 2020).
Factor 3: Civic knowledge and skills
Factor 3 contains five items that capture students’ knowledge of opportunities to get involved in their community and ability to help address community issues (see Table 6).
Factor 3 Survey Items and Analyses
| Survey item | Item level | Concept level | ||
|---|---|---|---|---|
| Mean ± SD (% disagree) (n = 419) | Content analysis | Factor loadings | Themes | |
| I feel confident that I will be able to apply what I have learned in my classes to solve real problems in society. | 6.76 ± 1.65 (8.11%) | Students described how their engineering courses helped them learn to address community needs. | 0.63 | Students were aware of opportunities to volunteer in their campus community and in their pre-college community. Students could describe issues affecting their community and propose solutions. |
| I have the professional knowledge and skills that I need to help address community issues. | 6.35 ± 1.61 (12.9%) | Students recognized how their developing professional skills and knowledge could help their community. | 0.78 | |
| I know a lot about opportunities to become involved in the community. | 6.11 ± 1.74 (17.2%) | Students were knowledgeable about opportunities to volunteer in their campus community. | 0.75 | |
| I am able to plan or help implement an initiative that improves the community. | 5.96 ± 1.78 (20.3%) | Some students provided examples of taking initiative to help their community. | 0.79 | |
| I would say that most other students know less about community organizations and volunteer opportunities than I do. | 5.04 ± 1.83 (38.7%) | On average, most students did not compare their personal community involvement with their peers’ involvement. | 0.52 | |
-
Note. Responses were on a 9-point Likert-type scale where 1 = Strongly Disagree and 9 = Strongly Agree.
How did students quantitatively respond to individual items?
Student responses to Factor 3 items were slightly positive, suggesting that students had some awareness of civic opportunities in college. Two survey items in Factor 3 related to students’ disciplinary knowledge and skills; these items had the highest agreement. The three remaining items showed relatively high levels of disagreement. Of particular note is the item that 38.7% of students disagreed with: “I would say that most other students know less about community organizations and volunteer opportunities than I do.” This item seems to prompt students to consider too many aspects simultaneously (e.g., other students, what other students know about community organizations, what other students know about volunteering opportunities, what “I” know about these things, and how what “I” know compares with what others know). Thus, we posit that many students struggled to interpret this question and that it ought to be simplified.
How and to what extent did interview responses align with survey item responses?
In both surveys and interviews, students recognized how engineering knowledge and skills could benefit community needs. We posit that many students were drawing on these considerations when responding to items. In addition, interviewees provided explicit examples of ways that they could apply knowledge from their courses to address community needs. For example, Julia, who was enrolled in a service-learning engineering course, explained:
This is my first experience of engineering and applying it to the community. Meeting the community partners that we work with and hearing them talk about their experience and why they need the product we’re working on, it showed me how the work I think I’m doing is contributing in the bigger picture.
Through service-learning, Julia learned how engineers could work alongside community partners to help address community needs. In addition, Isabella felt that she could apply her engineering learning to help her community:
These first-year engineering classes have been about general critical-thinking and programming and just really general concepts. Once I get into a job or a career, I’ll be doing exactly what they need, but I’ll be able to also retain some of the skills I’m not using. And potentially use them in a different area of volunteering, or some type of community involvement.
In what other ways did interviews provide nuance to the survey items?
Many students were knowledgeable about issues and opportunities within their communities. This was particularly evident among students who named specific community needs and potential solutions. However, few students were confident in their ability to address community issues. For example, Sara was not confident that she was well informed of issues on campus, but she was aware of the community’s need for an affordable grocery store:
I’m not very knowledgeable about problems at [this university]. I think one that I have noticed is food insecurity since there’s no grocery store very close to the campus. And at the cultural center I talked to, I brought up the [supermarket] that’s being built. They don’t seem to think that it’s going to be super affordable.
Since students were in their first year of college, their professional skills were still developing. Nonetheless, they recognized how engineers could use their skills within and outside of their professional work. Moreover, like Factor 2, these considerations suggest the importance of accounting for other variables that might affect student self-efficacy, such as having access to service opportunities, leveraging social networks, and using other forms of social capital.
How did students perceive this aspect of civic-mindedness with respect to the engineering profession?
Students recognized that engineers had specialized knowledge and skills that could be used to address community issues. For example, Leo, like many other students, recognized how engineers’ technical expertise and problem scoping skills could be applied to community improvement projects:
Engineers have very good technical problem-solving skills. They’re very analytical, they like to look at all the specifications of every single component that goes into the final project before they even decide to go forward in the project. Say a community wanted to build a new housing development: engineers could look at where the housing development is, what it’s going to impact, [and] they could put their analysis skills to use in providing feedback to leaders in their community.
How do the above findings align with engineering education literature?
Factor 3 represents one’s knowledge of community issues and how to address them. It connects to various efforts in engineering, particularly in humanitarian engineering (Hynes & Swenson, 2013), social justice (Riley, 2008), and engineering justice (Leydens & Lucena, 2018). One relevant example is the framework of engineering for and with people (Hynes & Swenson, 2013). While potentially a subtle distinction, engineering for people focuses on engineers as the specialized knower, whereas engineering with people suggests that engineers are individuals who bring their unique knowledge to community issues to collaborate in solving communal issues. Like Hynes and Swenson (2013), we question whether actual engagement with community members, potentially through service-learning contexts, might help students grapple with such considerations. Notably, this was a limited aspect of most of the community-oriented design projects that students engaged in during their first-year project-based design courses (see Appendix A).
Factor 4: Empathic interpersonal communication
Factor 4 includes four items that represent students’ abilities to navigate interpersonal relationships and have empathy towards others (see Table 7).
Factor 4 Survey Items and Analyses
| Survey item | Item level | Concept level | ||
|---|---|---|---|---|
| Mean ± SD (% disagree) (n = 419) | Content analysis | Factor loadings | Themes | |
| I am a good listener, even when peoples’ opinions are different from mine. | 7.52 ± 1.44 (4.8%) | Many students valued listening and respecting others’ opinions. | 0.80 | Students indicated that listening and respecting others’ opinions was important. Some students also valued seeking out and listening to diverse perspectives. |
| I am able to respond to others with empathy, regardless of their backgrounds. | 7.46 ± 1.37 (3.1%) | Empathy did not come up explicitly in interviews, but listening and communication did. | 0.73 | |
| I listen to others and understand their perspective on controversial issues. | 7.34 ± 1.30 (2.4%) | Students mentioned the importance of listening and understanding others’ opinions. The controversial issues that students mentioned were largely political. | 0.75 | |
| When members of my group disagree on how to solve a problem, I like to try to build consensus. | 7.26 ± 1.35 (2.4%) | Students did not mention examples of disagreements with their teams. | 0.64 | |
-
Note. Responses were on a 9-point Likert-type scale where 1 = Strongly Disagree and 9 = Strongly Agree.
How did students quantitatively respond to individual items?
Student responses to these items were mostly positive with few students disagreeing to the associated survey items, thus suggesting students felt confident they had the skills to navigate interpersonal relationships. The most positive response of all survey items was to the item “I am a good listener, even when peoples’ opinions are different from mine” (M = 7.52, SD = 1.44), followed closely by the item “I am able to respond to others with empathy, regardless of their backgrounds” (M = 7.46, SD = 1.37). Teaming, communication, listening, and empathy were a concerted focus of the first-year engineering program in which all students participated, and this instruction may have contributed to these positive responses.
How and to what extent did interview responses align with survey item responses?
In interviews, students raised the importance of listening to and respecting others’ viewpoints, though they did not mention empathy explicitly. Madison emphasized listening as a way to care for people in her community, stating: “I think a lot of people in the community need as much physical support with food and money as emotional support. So listening to your friends, active listening [is important].”
In addition to listening, students felt it was important to respect others’ opinions, especially when they differed from one’s own. Julia advised people to “speak out for whatever you believe in, and represent that idea in a good manner, not shaming people into joining you in your beliefs.” When conveying one’s opinions, Julia advocated for seeking to understand people who may disagree with you. Other students valued listening when discussing controversial issues, especially when those issues involved politics. For example, Larry talked about listening to people with differing political stances:
Everyone’s going to have a different opinion, so it is hard for people to get along. But I don’t think you have to get along, you just have to understand other people’s points of view and why they think that way.
Larry felt that having differing opinions should not preclude people from discussing and understanding others’ perspectives.
In what other ways did interviews provide nuance to the survey items?
In interviews, students emphasized the value of engaging with difference. Specifically, some students felt it was important to seek out and listen to diverse perspectives. Moreover, beyond listening to people with different viewpoints, Leo recognized the value of integrating diverse perspectives: “Having exposure to multiple different types of people, personalities, can help improve your own creative style and give you more ways to think or analyze different situations. … I actively seek to meet new people from many different backgrounds.”
Thus, Leo sought to engage people with different backgrounds and interests from him and valued their contributions to his thinking. He aspired to integrate diverse perspectives into his everyday life. Similarly, Julia appreciated how volunteering exposed her to people with different life experiences from her own:
I think that it’s important for everyone to do some volunteering work, because it does broaden your perspective and give you new insights by seeing what other people are going through. … [Volunteering] has exposed me to a lot of different people in the community. And getting to see their perspective and what they’re passionate about and their situation in life I think is really cool.
How did students perceive this aspect of civic-mindedness with respect to the engineering profession?
Interviewees recognized that collaboration and communication are important aspects of engineering. For example, Leo stated, “Engineers can solve important problems. And it doesn’t take too many people to solve a problem. It does take a team with lots of different viewpoints, but any sized team can achieve a goal in the end.” Leo explained that having a diverse team would help an engineering team work together more effectively.
While communication was an aspect of many responses, some students specifically recognized that listening was central to engineers’ job roles. John explained:
Engineers should listen to their clients. No matter which client I work with, I need to treat them well. It’s a service-based job, just like waiters, teachers; they all serve people. If you serve people, you should treat them well and do the right thing.
In addition to listening to one’s team members, John emphasized that engineers should listen to and value the opinions of people they serve.
How do the above findings align with engineering education literature?
Empathic interpersonal communication is important for engineers because they often work in teams and need to consider diverse viewpoints. Leydens and Lucena (2009) argued that listening was missing in engineering education, but they recognized that listening was necessary for effectively engaging community members. When engineers listen with empathy, they can integrate the perspectives and needs of community members into their work and empower community members with ownership over the project, which can improve the sustainability of the project (Leydens & Lucena, 2009). In addition, returning to the notion of engineering for and with community members (Hynes & Swenson, 2013), we argue that promoting civic-mindedness can involve teaching effective communication and collaboration skills to help students empathize with community members.
Factor 5: Civic intentions and obligations
Factor 5 includes four items that capture students’ intentions to stay informed of current events and participate in political processes (see Table 8).
Factor 5 Survey Items and Analyses
| Survey item | Item level | Concept level | ||
|---|---|---|---|---|
| Mean ± SD (% disagree) (n = 419) | Content analysis | Factor loadings | Themes | |
| It is important for me to vote and be politically involved. | 6.73 ± 2.02 (13.4%) | Many students indicated voting was important to them. | 0.63 | Students recognized the importance of voting and taking political action. Students wanted to stay informed, but some found it difficult to prioritize among academic demands on their time. |
| I intend to stay current with the local and national news. | 6.56 ± 1.87 (13.8%) | Many students followed the news, primarily national news and some local news. | 0.85 | |
| I stay up to date on the current political issues in the community. | 5.89 ± 2.06 (23.4%) | Students more often mentioned issues facing their campus community rather than the local community outside the university. | 0.81 | |
| I plan to participate in advocacy or political action groups after I graduate. | 5.05 ± 2.19 (39.9%) | Advocacy and political action groups were rarely mentioned. | 0.64 | |
-
Note. Responses were on a 9-point Likert-type scale where 1 = Strongly Disagree and 9 = Strongly Agree.
How did students quantitatively respond to individual items?
Average responses to all but one item were slightly positive, suggesting students intended to become engaged in civic and political issues. However, responses showed moderate disagreement compared with other factors, with more than 10% of respondents expressing disagreement to each item. The item with the highest disagreement, “I plan to participate in advocacy or political action groups after I graduate,” indicated that 40% of students were not planning to join advocacy or political action groups after graduation. This high disagreement could suggest these first-year engineering students are not yet interested in using political engagement to enact social change, or it may suggest that advocacy and political action groups are not salient to students’ future careers. Thus, based on these responses, when used in the context of engineering, these items may need to be rephrased or broadened to be more relevant to the engineering profession.
How and to what extent did interview responses align with survey item responses?
In interviews, students agreed it was important to stay informed and engage in political processes, though students varied in their perceptions regarding what level of involvement engineers should have. Some students became more interested in politics in college than in high school. For example, Sara became increasingly aware of social inequities and viewed politics as a path for social change:
I’m definitely more interested in politics now than I was as a high schooler. Just becoming more aware of the problems in America, everyone has someone in their life most likely that is affected by some pitfall in American society.
Conversely, other students noted that they were less involved with the news and politics in college. Isabella indicated she lacked the time to keep up with the news: “I find going online to search up things and find the correct news instead of the fake news, it just takes so much time out of my day. I don’t have that time anymore; I’m spending it studying.” Though Isabella wanted to follow current events, she found it difficult to devote her time to finding credible news sources. This challenge was common among the interviewed students.
In what other ways did interviews provide nuance to the survey items?
Though students recognized the importance of political engagement, they acknowledged barriers that made it difficult to become politically involved. Many students were deterred by political conflicts and sought to avoid the emotionally charged nature of political discussions. For example, Grace noted the mental toll of participating in politics:
[Politics is] emotionally exhausting. Honestly, I don’t know if I always have the emotional capacity to deal with it. … If it’s something that you genuinely feel is an important cause, you should have involvement in it. But if you cannot commit the mental faculties to being involved, then I feel like it’s not as much pressure as community involvement. Because it’s usually a larger scale.
Grace preferred to reserve her energy for forms of community engagement other than politics that were less emotionally taxing and resulted in a more tangible impact on her community. Like Grace, many students valued the impact of politics and wanted to be politically involved, though some students acknowledged they did not always follow through on their intentions to stay updated on contemporary news and politics issues.
The misalignment between perceived political importance and engagement was mirrored in the survey results, where students responded more affirmatively about their intent to stay current with news rather than their keeping up with local community issues. This discrepancy may be explained by students choosing to follow national news more closely than local news. Most students did not discuss local politics whatsoever. Even when students brought up issues in their local communities, they mostly discussed issues facing the campus community rather than the surrounding community outside the university.
How did students perceive this aspect of civic-mindedness with respect to the engineering profession?
We analyzed students’ beliefs about the relationality between engineering and politics and found a range of responses. Students readily described ways that politics influenced engineers, though they struggled to identify ways that engineers could influence politics. When asked about how politics and engineering relate, Leo explained:
They’re so very different. Unless you’re an engineer in politics, I’m not sure how you would influence politics that much, besides casting your vote and just talking with the people around you. … Politicians control funding for organizations that do a lot of engineering and science. … Politicians can use their control of money to focus engineering towards different sectors that they value.
Leo acknowledged the influence of politics on engineering, namely politicians prioritizing certain industry sectors with funding, though he was unsure about ways that engineers could engage in politics. Yet many other students recognized how engineers and politics could work together to address social issues. For example, Sara described how engineers and the government could collectively address climate change:
I think engineering right now should focus on the environment. Things need to change at a government level with regulations. … I think cleaning the environment and preventing more damage is where engineers need to focus more. I think making engineers to make these innovations, and getting the government to become stricter, go hand and hand.
Sara expressed her opinion that governments needed to impose stricter regulations and engineers needed to innovate for more sustainable technologies. Her example of climate change shows how societal challenges can benefit from engineers and politics working in tandem.
How do the above findings align with engineering education literature?
Factor 5 captures one’s intentions to be politically engaged inside or outside of engineering. Our findings suggest that engineering students faced barriers to their political involvement and felt politics and engineering were disconnected. This disconnect reflects a culture of depoliticization in engineering that demarcates engineering as a purely “technical” space detached from political considerations (Cech & Sherick, 2015). Students’ socialization into a culture of depoliticization, coupled with their discomfort with political discussions, suggest a concerning trend of students shying away from political engagement in their personal and professional lives. Morgan et al. (2020) identified challenges that engineering students face in their political involvement, including time constraints from academics and experiences that relegated politics to the periphery of engineering. Like Morgan et al. (2020), we recommend engineering instructors identify ways to help students engage meaningfully with politics within curriculum. By addressing politics within engineering curriculum, students can see examples of the connections between engineering and politics.
Limitations and Future Work
First, since civic-mindedness is a socially desirable disposition, we were cognizant of such possible biases in our data. Steinberg et al. (2011) did not find evidence of social desirability response bias in their validity study of the Civic-Minded Graduate Scale. Though we did not perform a similar quantitative test, in interviews, we verified participants’ accounts of their civic-minded beliefs and intentions by asking for specific examples and experiences related to their civic involvement. This triangulation suggested a potential disconnect between students’ civic intentions and behaviors that can be explored further, potentially by distinguishing between civic-mindedness and civic engagement in the design and administration of the instrument.
Second, we did not account for demographic differences. Most importantly, we did not hypothesize that the structure of the Civic-Minded Graduate Scale would differ by any demographics. Thus, while there may be significant differences by gender in students’ responses, we do not believe the underlying structure of the instrument’s factors would differ by gender. Moreover, we did not have sufficient sample sizes to confidently perform comparisons by race. An interesting topic for future work would be to investigate differences by gender and race and variation in students’ civic engagement and their perceptions of civic-mindedness.
Third, we did not sample interview participants based on their social identities and backgrounds. Our study sample is predominantly white and largely consists of students who seemingly had ample access to civic opportunities through their high schools and local communities. Thus, our findings do not account for the influences of different racial and economic backgrounds on civic engagement and development (Malin et al., 2017). As with race and gender, future work should explore variations in civic-mindedness by socioeconomic status.
Finally, our study focuses on first-year engineering students, which is both a limitation and strength of our study. Though our findings are not necessarily generalizable to all engineering students, we feel more confident that our findings are representative of first-year engineering students. Thus, the generalizability of our findings is narrower but more accurate for the specific population we studied (Jager et al., 2017). Future work ought to extend these findings with engineering students beyond their first year, and across disciplines, to understand how civic-mindedness manifests as students progress through their engineering programs.
Implications
In this study, we focused on first-year engineering students, but our findings have implications beyond engineering programs alone. We believe the findings can be useful for engineering faculty interested in civic learning as well as civic engagement programs and educators. Thus, this section begins by discussing implications for educating civic-minded engineers. Next, we consider strategies for assessing civic-mindedness in general.
Implications for Teaching and Engagement
This study provides considerations for using the Civic-Minded Graduate Scale to assess civic-related learning outcomes. Civic learning outcomes are closely related to numerous professional skills important in engineering education, such as the ability to identify the broader (i.e., “macro” and “political”) impacts of engineering, interpersonal communication and collaboration, and social and professional responsibility (ABET, 2021; Shuman et al., 2005). Thus, civic engagement as a pedagogical approach can provide a rich mechanism for helping students develop the professional skills required of engineering programs. These considerations align with findings from Wittig (2013) indicating that students who engaged in Engineers Without Borders, compared with other engineering coursework, perceived significantly greater learning gains associated with ethics; awareness of broader impacts of engineering; and awareness of the roles of engineers in business, policy, and leadership. While Wittig (2013) employed the ABET EC 2000 learning outcomes, other instructors might instead use, or combine Wittig’s approach with, the five factors described above, with potential additions as suggested in the next section.
Implications for Research and Evaluation
For those who wish to use the Civic-Minded Graduate Scale or similar instruments to assess civic outcomes, we offer reflections on our methods and associated recommendations for research and evaluation.
1. When administering survey instruments about civic outcomes, prime students to respond with a specific context in mind.
When we administered the Civic-Minded Graduate Scale, we did not give students a particular frame or context to consider as they responded to the survey items. Thus, while some items prompted students to reflect on community, we cannot be certain how survey respondents defined community when responding to these items. However, in interviews, we noticed variations in how students scoped their community. Many students identified with multiple communities, including their hometowns, residence halls, the campus community, and the local community. We recommend asking students to consider a specific community context when responding to the survey, especially on items pertaining to civic attitudes and behaviors. Priming responses to focus on a specific community keeps students’ responses consistent and is thus potentially more useful and valid for comparison tests and other analyses.
Context is also important to consider because civic outcomes can manifest differently based on one’s environment. For example, prior studies have shown that civic engagement decreases as students graduate from high school and transition into early adulthood (Finlay et al., 2010; Malin et al., 2017). These changes in civic participation have been attributed in part to changes in students’ community context, such as the loss of supportive structures (i.e., family and social groups) and increasing barriers (i.e., academic demands on time) (Finlay et al., 2010; Malin et al., 2017). By focusing on a specific context, we can understand factors in a particular environment that promote or inhibit civic involvement.
2. Consider utilizing mixed methods to assess civic learning outcomes.
In this study, we outlined a mixed methods approach to validating the Civic-Minded Graduate Scale. This approach can be applied to other instruments in a variety of contexts at the course, department, or university levels. We reflect on our methods to offer some insights into using mixed methods for assessing civic outcomes. This study began with factor analyses to generate five constructs explaining the underlying structure of civic-mindedness as measured by the Civic-Minded Graduate Scale. We then conducted interviews to gather qualitative evidence on how well these five constructs accounted for civic-mindedness within the context of first-year engineering students.
The interviews revealed some limitations of relying on survey data alone. While the quantitative data helped us understand trends in students’ attitudes and beliefs, the qualitative data revealed important nuances in students’ interpretations of civic-mindedness. Furthermore, the Civic-Minded Graduate Scale focuses largely on students’ intentions, attitudes, and beliefs towards civic engagement rather than their behavior. The qualitative data captured environmental factors that explained why civic-minded dispositions did not necessarily translate into civic engagement. Though many students believed it was important to be involved in their communities, they were generally not as involved as they hoped to be. The most common barrier to involvement was the espoused pressure to prioritize academics over service. Interviewees who remained most civically engaged integrated service with academics through service-learning or social groups. Though we acknowledge mixed methods research adds time and resources to a study, these findings indicate how meaningfully engaging distinct data sources can lead to significant insights (Greene, 2007). Indeed, the Civic-Minded Graduate Scale was initially developed and validated using multiple forms of quantitative and qualitative evidence. Thus, we encourage researchers interested in civic-mindedness to collect qualitative data alongside use of the Civic-Minded Graduate Scale in future research.
Notes
- In the original study introducing the five-factor structure (see Hess et al., 2021), we named Factor 2 “Confidence in building consensus.” One reviewer made an apt observation that led us to revisit how we originally named Factor 2. Along with our colleagues in Hess et al. (2021), we reviewed the items in Factor 2 and settled on a new name that we believe more accurately represents the four items: “Confidence in influencing social issues.” The items in Factor 2 remain the same.
Acknowledgments
We thank the students who shared their time and insights as participants in our study and the instructors who helped us recruit participants. We also thank the reviewers and editors for their constructive feedback and guidance on revisions for this manuscript. We also acknowledge the National Science Foundation for support under the Graduate Research Fellowship Program (GRFP) under grant number DGE-1842166. The opinions, findings, and conclusions or recommendations expressed are those of the authors and do not necessarily reflect the views of the National Science Foundation.
References
ABET. (2021). Criteria for accrediting engineering programs, 2021–2022. https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2021-2022/https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2021-2022/
Bauer, T., Kniffin, L. E., & Priest, K. L. (2015). The future of service-learning and community engagement: Asset-based approaches and student learning in first-year courses. Michigan Journal of Community Service Learning, 22(1), 89–92.
Beever, J., & Brightman, A. O. (2016). Reflexive principlism as an effective approach for developing ethical reasoning in engineering. Science and Engineering Ethics, 22(1), 275–291. https://doi.org/10.1007/s11948-015-9633-5https://doi.org/10.1007/s11948-015-9633-5
Bielefeldt, A. R., & Canney, N. (2014). Impacts of service-learning on the professional social responsibility attitudes of engineering students. International Journal for Service Learning in Engineering: Humanitarian Engineering and Social Entrepreneurship, 9(2), 47–63. https://doi.org/10.24908/ijsle.v9i2.5449https://doi.org/10.24908/ijsle.v9i2.5449
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. httxsps://doi.org/10.1191/1478088706qp063oahttxsps://doi.org/10.1191/1478088706qp063oa
Bringle, R. G., Hahn, T. W., & Hatcher, J. A. (2019). Civic-minded graduate: Additional evidence II. Advances in Theory and Methodology, 7(1). https://doi.org/10.37333/001c.11481https://doi.org/10.37333/001c.11481
Bringle, R. G., & Wall, E. (2020). Civic-minded graduate: Additional evidence. Michigan Journal of Community Service Learning, 26(1). https://doi.org/10.3998/mjcsloa.3239521.0026.101https://doi.org/10.3998/mjcsloa.3239521.0026.101
Canney, N., & Bielefeldt, A. (2015). A framework for the development of social responsibility in engineers. International Journal of Engineering Education, 31(1), 414–424.
Cech, E. A., & Sherick, H. M. (2015). Depoliticization and the structure of engineering education. In Christensen, S. H., Didier, C., Jamison, A., Meganck, M., Mitcham, C., & Newberry, B. (Eds.), International perspectives on engineering education (pp. 203–216). Springer.
Dall’Alba, G. (2009). Learning professional ways of being: Ambiguities of becoming. Educational Philosophy and Theory, 41(1), 34–45. https://doi.org/10.1111/j.1469-5812.2008.00475.xhttps://doi.org/10.1111/j.1469-5812.2008.00475.x
Dym, C. L., Agogino, A. M., Eris, O., Frey, D. D., & Leifer, L. J. (2005). Engineering design thinking, teaching, and learning. Journal of Engineering Education, 94(1), 103–120. https://doi.org/10.1002/j.2168-9830.2005.tb00832.xhttps://doi.org/10.1002/j.2168-9830.2005.tb00832.x
Ehrlich, T. (2000). Preface. In Ehrlich, T. (Ed.), Civic responsibility and higher education (pp. v–x). The American Council on Education and the Oryx Press.
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.xhttps://doi.org/10.1111/j.1365-2648.2007.04569.x
Fila, N. D., Hess, J. L., & Purzer, S. (2015, June 14–17). A dialectic data integration approach for mixed methods survey validation [Paper presentation]. 122nd ASEE Annual Conference and Exposition, Seattle, WA, United States. https://doi.org/10.18260/p.23376https://doi.org/10.18260/p.23376
Finlay, A., Wray-Lake, L., & Flanagan, C. (2010). Civic engagement during the transition to adulthood: Developmental opportunities and social policies at a critical juncture. In Sherrod, L. R., Torney-Purta, J., & Flanagan, C. A. (Eds.), Handbook of research on civic engagement in youth (pp. 277–305). John Wiley & Sons. https://doi.org/10.1002/9780470767603.ch11https://doi.org/10.1002/9780470767603.ch11
Greene, J. C. (2007). Mixed methods in social inquiry. Jossey-Bass.
Herkert, J. R. (2005). Ways of thinking about and teaching ethical problem solving: Microethics and macroethics in engineering. Science and Engineering Ethics, 11, 373–385.
Hess, J. L., Lin, A., Fore, G. A., Hahn, T., & Sorge, B. (2021). Testing the Civic-Minded Graduate Scale in science and engineering. International Journal of Engineering Education, 37(1), 44–64.
Hunter, M. S., & Moody, B. L. (2009). Civic engagement in the first college year. In Jacoby, B. & Associates (Eds.), Civic engagement in higher education: Concepts and practices (pp. 69–84). Jossey-Bass.
Hynes, M., & Swenson, J. (2013). The humanistic side of engineering: Considering social science and humanities dimensions of engineering in education and research. Journal of Pre-College Engineering Education, 3(2), 31–42.
Jager, J., Putnick, D. L., & Bornstein, M. H. (2017). More than just convenient: The scientific merits of homogeneous convenience samples. Monographs of the Society for Research in Child Development, 82(2), 13–30. https://doi.org/10.1111/mono.12296https://doi.org/10.1111/mono.12296
Jing, S., & Doorn, N. (2020). Engineers’ moral responsibility: A Confucian perspective. Science and Engineering Ethics, 26(1), 233–253. https://doi.org/10.1007/s11948-019-00093-4https://doi.org/10.1007/s11948-019-00093-4
Leydens, J. A., & Lucena, J. C. (2009). Listening as a missing dimension in engineering education: Implications for sustainable community development efforts. IEEE Transactions on Professional Communication, 52(4), 359–376.
Leydens, J. A., & Lucena, J. C. (2018). Engineering justice: Transforming engineering education and practice. IEEE Press.
Lin, A., & Hess, J. L. (2021, July 26). Developing a framework for civic responsibility in engineering education [Paper presentation]. ASEE Annual Conference and Exposition, Virtual.
Lynch, W. T., & Kline, R. (2000). Engineering practice and engineering ethics. Science, Technology, & Human Values, 25(2), 195–225. https://doi.org/10.1177/016224390002500203https://doi.org/10.1177/016224390002500203
Malin, H., Han, H., & Liauw, I. (2017). Civic purpose in late adolescence: Factors that prevent decline in civic engagement after high school. Developmental Psychology, 53(7), 1384–1397. https://doi.org/10.1037/dev0000322https://doi.org/10.1037/dev0000322
Manning-Ouellette, A., & Hemer, K. M. (2019). Service-learning and civic attitudes: A mixed methods approach to civic engagement in the first year of college. Journal of Community Engagement and Higher Education, 11(3), 5–18.
Mayhew, M. J., & Engberg, M. E. (2011). Promoting the development of civic responsibility: Infusing service-learning practices in first-year “success” courses. Journal of College Student Development, 52(1), 20–38. https://doi.org/10.1353/csd.2011.0007https://doi.org/10.1353/csd.2011.0007
Messick, S. (1995). Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and Practice, 14(4), 5–8. https://doi.org/10.1111/j.1745-3992.1995.tb00881.xhttps://doi.org/10.1111/j.1745-3992.1995.tb00881.x
Morgan, D. L., Davis, K. B., & López, N. (2020). Engineering political fluency: Identifying tensions in the political identity development of engineering majors. Journal of Engineering Education, 109(1), 107–124. https://doi.org/10.1002/jee.20300https://doi.org/10.1002/jee.20300
Peters, S. J. (2004). Educating the civic professional: Reconfigurations and resistances. Michigan Journal of Community Service Learning, 11, 47–58.
Reason, R. D., & Hemer, K. (2015). Civic learning and engagement: A review of the literature on civic learning, assessment, and instruments. Association of American Colleges and Universities, Degree Qualifications Profile, Civic Learning Task Force.
Riley, D. (2008). Engineering and social justice. Morgan & Claypool.
Rottmann, C., & Reeve, D. (2020). Equity as rebar: Bridging the micro/macro divide in engineering ethics education. Canadian Journal of Science, Mathematics and Technology Education, 20, 146–165.
Scherrer, C., & Sharpe, J. (2020). Service learning versus traditional project-based learning: A comparison study in a first year industrial and systems engineering course. International Journal for Service Learning in Engineering: Humanitarian Engineering and Social Entrepreneurship, 15(1), 18–32. https://doi.org/10.24908/ijsle.v15i1.13569https://doi.org/10.24908/ijsle.v15i1.13569
Schneider, J., Leydens, J. A., & Lucena, J. (2008). Where is “community”? Engineering education and sustainable community development. European Journal of Engineering Education, 33(3), 307–319. https://doi.org/10.1080/03043790802088640https://doi.org/10.1080/03043790802088640
Shelby, R., Ansari, F., Patten, E., Pruitt, L., Walker, G., & Wang, J. (2013). Implementation of leadership and service learning in a first-year engineering course enhances professional skills. International Journal of Engineering Education, 29(1), 1–14.
Shuman, L. J., Besterfield-Sacre, M., & McGourty, J. (2005). The ABET “professional skills”—Can they be taught? Can they be assessed? Journal of Engineering Education, 94(1), 41–55. https://doi.org/10.1002/j.2168-9830.2005.tb00828.xhttps://doi.org/10.1002/j.2168-9830.2005.tb00828.x
Steinberg, K. S., Hatcher, J. A., & Bringle, R. G. (2011). Civic-minded graduate: A north star. Michigan Journal of Community Service Learning, 18(1), 19–33.
Terry, G., Hayfield, N., Clarke, V., & Braun, V. (2017). Thematic analysis. In Willig, C. & Stainton-Rogers, W. (Eds.), The SAGE handbook of qualitative research in psychology (2nd ed.). SAGE Publications.
Torney-Purta, J., Cabrera, J. C., Roohr, K. C., Liu, O. L., & Rios, J. A. (2015). Assessing civic competency and engagement in higher education: Research background, frameworks, and directions for next-generation assessment. ETS Research Report Series, 2015(2), 1–48. https://doi.org/10.1002/ets2.12081https://doi.org/10.1002/ets2.12081
Tsang, E. (2000). Introduction. In Tsang, E. (Ed.), Projects that matter: Concepts and models for service-learning in engineering (pp. 1–12). American Association for Higher Education.
Walther, J., Miller, S. E., & Sochacka, N. W. (2017). A model of empathy in engineering as a core skill, practice orientation, and professional way of being. Journal of Engineering Education, 106(1), 123–148. https://doi.org/10.1002/jee.20159https://doi.org/10.1002/jee.20159
Wittig, A. (2013). Implementing problem based learning through Engineers Without Borders student projects. Advances in Engineering Education, 3(4), 1–20.
Author Bios
Athena Lin is a graduate student in the School of Engineering Education at Purdue University and a National Science Foundation Graduate Research Fellow. She received her BS in Materials Science and Engineering from the University of Illinois at Urbana–Champaign. Her research interests center on engineering ethics education and preparing students for responsible engineering practice.
Justin L. Hess is an Assistant Professor in the School of Engineering Education at Purdue University. Dr. Hess received each of his degrees from Purdue University, including a PhD in Engineering Education, a Master of Science in Civil Engineering, and a Bachelor of Science in Civil Engineering. His primary research interests focus on empathy; engineering ethics; engineering design; and diversity, equity, and inclusion.
APPENDIX A Overview of First-Year Engineering Course
How did the course fit into the overall engineering curriculum of the university?
Students at the university are required to participate in a common first-year curriculum. During their first semester, most students take a standard first-year engineering course sequence wherein the first course emphasizes data analytics and design and the second course emphasizes computational thinking. In this study, all survey participants engaged in the standard first semester course. In addition, two interviewees were enrolled in a service-learning course for engineers.
What did the students do with community partners?
The standard first-year engineering course engages over 2,000 students each year, and these students participate in “sections” with up to 120 students. Each section is led by a different instructor, although some instructors lead multiple sections. Instructors have the freedom to decide what design project students will engage in. Design projects are not required to have a service-learning or community engagement aspect. Survey responses came from four instructors across six sections, and students in these sections completed one of the following design projects:
Engaging with an industry partner to develop a waste sorting system for use in a local basketball arena—a representative of the industry partner visited the course at multiple stages and throughout the project; students could share questions with the industry partner via the course instructor.
Developing toys for two- to four-year-old children in partnership with a local organization. While students needed to consider differently abled children, they were not required to design for them—students interviewed teachers during the design stage and then presented functional prototypes to children.
Designing campus infrastructure to increase accessibility—students engaged in this design project may or may not have contacted members of the campus community, including designers/overseers of select aspects of infrastructure or users of infrastructure (although most students were users of the selected topic).
How were they supported and prepared to engage in service?
As indicated above, the nature of the “service” varied by design project. In design project 1, students’ interaction with an industry representative was largely facilitated and monitored by course instructors. In design project 3, students had freedom and autonomy to interact with community partners or users but were not required to.
What were the primary learning objectives of the course?
Students’ final design project included the following learning objectives:
DV01 - Efficient use of engineering tools for basic statistics. Use of automated solutions, such as cell referencing and built-in functions.
DV02 - Select appropriate graphical representation of data set based on data characteristics, such as numerical (discrete or continuous) or categorical (ordinal or nominal).
DV04 - Prepare a table for technical presentation with proper formatting.
DV05 - Prepare a chart for technical presentation with proper formatting, including title, axes labels, appropriately scaled axes, units, and appropriate markers.
EB01 - Test prototypes and analyze results to inform comparison of alternative solutions.
EB02 - Identify assumptions made in cases when there are barriers to accessing information.
EB03 - Clearly articulate reasons for answers with explicit reference to data to justify decisions or to evaluate alternative solutions.
EB04 - Justify chosen metrics and the corresponding assigned weights to evaluate potential solutions, based on stakeholder needs.
EB05- Present findings from iterative testing or optimization efforts used to further improve aspect or performance of a solution.
EB06 - Clearly articulate reasons for answers when making decisions or evaluating alternative solutions.
EE02- Predict/identify the potential ethical dilemmas and consequences that result from implementing solutions.
IF01 - Generate a wide range of solutions including ideas not readily obvious or combinations of ideas in new ways.
IF02 - Explicitly use and document two or more ideation strategies (biomimicry, brainstorming, exploration of prior art, etc.) to generate ideas.
IF03 - Generate testable prototypes for a set of potential solutions.
IL01 - Ask questions to determine what new information is needed to scope and solve a problem.
IL04 - Include citations within the text (in-text citations) that show how the references at the end of the text are used as evidence to support decisions.
IL05 - Format reference list of used sources that is traceable to original sources (American Psychological Association style is recommended).
PA01 - Identify strengths in problem-solving/design approach.
PA02 - Identify limitations in the approach used.
PA03 - Identify potential behaviors to improve approach in future problem-solving/design projects.
PC02- Use professional communication (written, visual, and oral), free of grammatical or spelling mistakes and in a formal tone, appropriate for engineering school and workplace.
PC03 - Present all visuals with captions (e.g., figure number, table number, and brief description).
PC04 - Professionally present all visual representations (figures, images, sketches, or prototypes) to clearly convey meaning by labeling key components to show their form and function.
PC05 - Fully address all parts of assignment by following instructions and completing all work.
PS01 - Explain the problem based on synthesis of client, user, and other stakeholder needs.
PS02 - Justify why problem is important to solve by making reference to relevant global, societal, economic, or environmental issues.
PS03 - Explain key specifications (in terms of criteria and constraints) that address what the client wants and what the user needs.
PS04 - Identify potentially competing or conflicting needs.
SQ01 - Use accurate, scientific, mathematical, and/or technical concepts, units, and/or data in solutions.
SQ02 - Justify design solution based on how well it meets criteria and constraints.
SQ03- To justify qualities of a solution and recognize any limitations, explain the trade-offs made to arrive at a final solution.
SQ04 - Present a succinct and clear value proposition that highlights novel aspects of the solution, including key benefits.
TW02 - Document all contributions to the team performance with evidence that these contributions are significant.
How was reflection deployed?
While design projects varied, there was a common set of milestones for those that students completed in teams of three or four. Milestones included (1) problem scoping; (2) idea generation; (3) peer evaluation; (4) prototyping, testing, and weighted-decision matrix; (5) preliminary presentation; (5) iteration; (6) final report; and (7) final presentation. All students were provided with a common template to work from to achieve each milestone. Reflections throughout the design project prompted students to consider whether they have provided evidence of attainment of the above learning objectives. In addition, students reflected on the overall quality of their design project, which included user considerations such as their needs, usability of the solution, and how to improve the solution in the future.
APPENDIX B Civic-Minded Graduate Scale Items
| Variable | Description | Codinga |
|---|---|---|
| KVO_01 | I know a lot about opportunities to become involved in the community. | 1 = Strongly Disagree 9 = Strongly Agree |
| KAK_01 | I am able to plan or help implement an initiative that improves the community. | 1 = Strongly Disagree 9 = Strongly Agree |
| SD_01 | I appreciate how my community is enriched by having some cultural or ethnic diversity. | 1 = Strongly Disagree 9 = Strongly Agree |
| KAK_02 | I have the professional knowledge and skills that I need to help address community issues. | 1 = Strongly Disagree 9 = Strongly Agree |
| BI_01 | I intend to stay current with the local and national news. | 1 = Strongly Disagree 9 = Strongly Agree |
| SCB_01 | I have often been able to persuade others to agree with my point of view. | 1 = Strongly Disagree 9 = Strongly Agree |
| KVO_02 | I am very familiar with clubs and organizations that encourage and support community involvement for college students. | 1 = Strongly Disagree 9 = Strongly Agree |
| SL_01 | I listen to others and understand their perspective on controversial issues. | 1 = Strongly Disagree 9 = Strongly Agree |
| DSE_01 | I can contribute to improving life in my community. | 1 = Strongly Disagree = Strongly Agree |
| KAK_03 | I feel confident that I will be able to apply what I have learned in my classes to solve real problems in society. | 1 = Strongly Disagree 9 = Strongly Agree |
| DSTK_01 | I want to dedicate my career to improving society. | 1 = Strongly Disagree 9 = Strongly Agree |
| DVCE_01 | I like to be involved in addressing community issues. | 1 = Strongly Disagree 9 = Strongly Agree |
| KCSI_01 | I stay up to date on the current political issues in the community. | 1 = Strongly Disagree 9 = Strongly Agree |
| DVCE_02 | I would say that the main purpose of work is to improve society through my career. | 1 = Strongly Disagree 9 = Strongly Agree |
| KVO_03 | I would say that most other students know less about community organizations and volunteer opportunities than I do. | 1 = Strongly Disagree 9 = Strongly Agree |
| SL_02 | I am a good listener, even when peoples’ opinions are different from mine. | 1 = Strongly Disagree 9 = Strongly Agree |
| BI_02 | I plan to participate in advocacy or political action groups after I graduate. | 1 = Strongly Disagree 9 = Strongly Agree |
| SD_02 | I am able to respond to others with empathy, regardless of their backgrounds. | 1 = Strongly Disagree 9 = Strongly Agree |
| BI_03 | I intend to be involved in volunteer service after I graduate. | 1 = Strongly Disagree 9 = Strongly Agree |
| DSTK_02 | I feel a deep conviction in my career goals to achieve purposes that are beyond my own self-interest. | 1 = Strongly Disagree 9 = Strongly Agree |
| KCSI_02 | I am prepared to write a letter to the newspaper or community leaders about a community issue. | 1 = Strongly Disagree 9 = Strongly Agree |
| KCSI_03 | I am aware of a number of community issues that need to be addressed. | 1 = Strongly Disagree 9 = Strongly Agree |
| DSE_02 | I am convinced that social problems are not too complex for me to help solve. | 1 = Strongly Disagree 9 = Strongly Agree |
| SCB_02 | Other students who know me well would describe me as a person who can discuss controversial social issues with civility and respect. | 1 = Strongly Disagree 9 = Strongly Agree |
| DSTK_03 | I believe that I have a responsibility to use the knowledge that I have gained to serve others. | 1 = Strongly Disagree 9 = Strongly Agree |
| DVCE_03 | I have a sense of who I am, which includes a sincere desire to be of service to others. | 1 = Strongly Disagree 9 = Strongly Agree |
| DSE_03 | I believe that having an impact on community problems is within my reach. | 1 = Strongly Disagree 9 = Strongly Agree |
| SCB_03 | When members of my group disagree on how to solve a problem, I like to try to build consensus. | 1 = Strongly Disagree 9 = Strongly Agree |
| SD_03 | I prefer to work in settings in which I interact with people who are different from me. | 1 = Strongly Disagree 9 = Strongly Agree |
| DVCE_04 | It is important for me to vote and be politically involved. | 1 = Strongly Disagree 9 = Strongly Agree |
-
a Survey items were administered on a Likert scale from 1 to 9. Only the extremes were labeled: 1 = Strongly Disagree and 9 = Strongly Agree.
APPENDIX C Five Constructs of the Civic-Minded Graduate Scale
| Factor | Construct description | Survey items | α |
|---|---|---|---|
| Factor 1 | Valuing community engagement | DSTK_01 DSTK_03 DVCE_01 DVCE_02 |
.90 |
| Factor 2 | Confidence in building consensus | DSE_02 DSE_03 SCB_01 SCB_02 |
.83 |
| Factor 3 | Civic knowledge and skills | KAK_01 KAK_02 KAK_03 KVO_01 KVO_03 |
.8 |
| Factor 4 | Empathic interpersonal communication | SCB_03 SD_02 SL_01 SL_02 |
.83 |
| Factor 5 | Civic intentions and obligations | BI_01 BI_02 DVCE_04 KCSI_01 |
.70 |