Author: Jeong-Hin Chin (University of Michigan)
How to Cite: Chin, J. (2021) “Exploring Quietness in Teams with Bootstrap Analysis”, University of Michigan Undergraduate Research Journal. 15(0). doi: https://doi.org/10.3998/umurj.1333
Team-based learning is a type of collaborative learning that is increasingly prevalent throughout all disciplines in higher education (Espey 24; Kim et al. 225–226). It is a common practice for faculty members to include team-based learning into an undergraduate curriculum, and for some, into a postgraduate curriculum. Inclusion of teamwork and team-based assignments can be commonly found in engineering, business, and social sciences programs across the world. Specifically, some faculty members will include team-based assignments in first-year introductory courses as well as final-year capstone projects. Inclusion of teamwork is considered to be beneficial to students in terms of learning to be good team members and this is why Gardner and Korth mentioned that “To remain innovative and competitive, businesses are looking for employees who can work and learn effectively in teams” (28). Besides, previous studies have shown that by learning in teams, students’ academic achievement and self-efficacy may increase.
Nonetheless, team-based learning is not a universally positive experience for all students, as some of the obstacles in teamwork include communication difficulties, uneven work allocation, free-riders and unfair grading experiences. (Wilson et al. 794; Pfaff and Huddleston 38; Medaille and Usinger 240–42). As teams are often made up of students who come from different backgrounds, it is normal for them to worry about potential obstacles in team-based learning. For example, all ten participants interviewed as part of Medaille and Usinger’s study noted that they had negative experiences with team-based collaborative learning due to the presence of free riders in their groups (Medaille and Usinger 246). In another study, non-high-achieving students were found to have difficulties in expressing their ideas, as high-achieving students in the groups had prepared for the projects on their own and were more likely to persuade the other students to follow their ideas rather than negotiating to resolve any conflicts. (Lee et al. 423; Lee and Lim 222)
In order to combat the perceived inequalities and negative experiences expressed by students regarding team-based projects, faculty members have introduced and developed techniques to assist in the teamwork process. (Chin et al. 3). Both the use of computer-supported collaborative learning environments and the use of peer mentors to assist teams throughout the semester have shown beneficial outcomes for students on teams. (Chin et al. 4; Ruël et al. 17–18). Teamwork assessment and support tools such as CATME encourage students to rate their teammates and themselves, while instructors can easily retrieve large amounts of data gathered by the tools. (Beigpourian et al. 11; Chin et al. 5; Maneeratana and Sripakagorn 5). With the help of these tools, instructors are able to look at one of the main aspects of a student’s negative experience of students in team-based learning, such as communication difficulties, and try to help the team out if there are disruptions in team dynamics (Beigpourian et al. 11)
Communication difficulties in teams can be due to students being reluctant to share their thoughts or just being shy and introverted. These may be the reasons why students are quiet in teams, but other reasons for being silent are often the result of personal, social, academic, cultural and contextual constraints (Medaille and Usinger). While previous studies have explored quiet students’ behaviors (Jin) and how they perceive themselves in collaborative learning (Medaille and Usinger), this study was designed to understand the following research questions (RQ):
RQ 1: To investigate if there is a relationship between three variables measuring various communication-related metrics on a beginning-of-term survey: “Extraversion” (self-rating of a student speaking up in groups), “BT_BelongingConcern” (students’ beginning of term concern regarding fit in the course), & the variable “SpeakUp” (self-rating of how likely they will hold back ideas to ensure other group members stay happy).
RQ 2: To investigate whether there is a relationship between self-rated previous team experiences (number and positive/negative valence) for the three variables mentioned. It is crucial for instructors to understand these two questions so that they can divide the students into groups that fit the students’ personality and traits based on the students’ responses to a survey administered at the beginning of term, rather than randomly grouping students. In order to test the research questions mentioned above, I propose the following hypothesis to be investigated in this study:
Hypothesis 1: There are significant correlations between students’ self-rated likelihood of speaking up in groups and (Extraversion), students’ beginning of term belonging concern score (BT_BelongingConcern), as well as between students’ self-rated likelihood of speaking up in groups and students’ self-rated likelihood of holding back ideas to ensure other group members stay happy (SpeakUp).
It is common to have students who tend to listen more and speak less in a group. These students are considered quiet and will often express agreement with the thoughts of others regardless of whether they actually agree with the ideas (Medaille and Usinger 242; Avoiding Communication 149–53). Moreover, students who talk a lot are seen to “dominate the interaction of the group” and “be quite willing to disagree with other group members” (Avoiding Communication 149–53). Medaille and Usinger also mentioned that quiet students suffer from “tensions between speaking and silence, engaging and disengaging and belonging and isolation when interacting with group members.” (254). I believe that a student’s initial perception towards a course before the term has begun will have an effect on how the students perceive their extraversion score in the course. Thus, the null and alternate hypotheses are:
While previous studies find these relationships (Medaille and Usinger 254), it is important to reproduce this finding quantitatively. Thus, by calculating the correlation between variables to find the R-squared value, we are able to be more confident in saying that there is a relationship between “Extraversion” & “BT_BelongingConcern” and “Extraversion” & “SpeakUp” among the students who responded to the survey.
I will be using Kendall’s tau-b (τb) statistic to calculate the correlation between the variables mentioned above. τb is chosen over other statistics to calculate the correlation as τb is the better in calculating the correlation between two ordinal variables (Khamis 159) as the questions in the survey were in the form of seven-point scales with identified end points.
Hypothesis 2a: A student with many previous teamwork experiences is more likely to score high in “Extraversion”, high in “SpeakUp,” and low in “BT_BelongingConcern.”
Hypothesis 2b: A student with past positive teamwork experiences is more likely to score high in “Extraversion”, high in “SpeakUp,” and low in “BT_BelongingConcern.”
For the two hypotheses above, I believe that when a student has many teamwork experiences or has had prior good experience with teamwork, they are able to recognize the benefit obtained through collaborative learning in groups such as increased individual achievement and persistence when facing adversity (Pfaff and Huddleston 38). Many prior teamwork experiences might have given students the confidence and more understanding of what to expect from a class that contains team projects while prior positive experiences might make the students want to work as a team again. Thus, both of the 2a and 2b hypotheses will use the same null and alternative hypotheses but will be tested using different smaller filtered data sets (as will be explained in the Methods section) to fit the condition of Much Experience vs. Less Experience and Positive Experience vs. Negative Experience.
After testing the three hypotheses mentioned above, I will continue this paper by calculating which predictors have a lower loss in predicting the variable “Extraversion” (rate of a student speaking up in groups) and will also perform cluster analysis on the students to facilitate team formation in the future.
The data for this study was collected from 2088 students enrolling in Engineering, Business, Informatics, and Architecture courses at the University of Michigan using a team assessment tool. The students answered a Beginning of Term survey before they were put into groups by their respective instructors. Therefore, the survey used in this research will collect the students’ personality and traits before the semester started and before they were divided up into teams. Although the students are from different courses, a similarity between these students is that the courses are conducted in a team-based collaborative learning format. In each course, students are required to complete project(s) assigned by the instructors in their respective teams. Projects vary by courses, but most of the projects require students to brainstorm ideas, solve challenges, and present their findings or products.
The survey forms contained different types of questions that vary from courses to courses. Among the questions included in the initial survey, 13 of them are the same across the 17 different courses. The final cleaned data is stored in a single file containing only the responses to the 13 questions. Note that while the team assessment tool included more questions and other assignments that students had to complete each week, only five variables from the responses are studied for the purposes of this project. In the original data file, six participants that contain NA values in one or more variables were removed in order to prevent error from occurring, leaving us with 2082 responses. Since the total number of samples collected was 2088, the 6 samples removed will not affect the computation in any important way. Some basic analysis on the five variables can be seen in Table 1, Table 2 and Figure 1.
Figure 3 to 5 tells us about the distribution of students’ responses in the three different variables. Nonetheless, these plots are not interesting by themselves as they do not tell us about the interaction between the variables. Therefore, to get a clearer picture of how the other two variables interact with Extraversion, two density plots were plotted to look at the distribution of students in “Extraversion v.s. SpeakUp” and “Extraversion v.s. BT_BelongingConcern”.
For this study, I used the “Beginning of Team” (BoT) survey which comprised a total of N = 2082 students (6 students’ responses were removed as mentioned above). The students responded to the following survey questions at the beginning of the Fall 2020 semester. Ordinal data was converted into numerical data so that computation can be carried out easily. I analyzed all the following variables in the range of 1–7 (or 5 in some cases) without performing any modification such as mid-ranking as I believed the students had clearly expressed their opinion using the Likert scales. The variables that were studied in this paper and its details are as:
[Extraversion] Students rated themselves on a 7-point scale to this statement. 1-point for this statement translates to “In groups, I tend to listen more than speak” and a 7-point translates to “I often speak up in groups”.
[SpeakUp] Students rated themselves on a Likert 7-point scale to this statement. 1-point for this statement translates to “I’d rather hold back ideas or preferences if my group stays happy” and a 7-point translates to “It’s easy for me to speak up about my ideas or preferences even if it disrupts my group”.
[BT_BelongingConcern] Students rated themselves on a Likert 7-point scale to this statement. 1-point for this statement translates to “I expect to fit right into the course” and a 7-point translates to “I expect to feel pretty out of place in the course”. Note that BT stands for Before Term, so this metric is about how the students feel about themselves fitting into the course before the class has actually begun.
[BT_PastGroups] Students answered this statement which asks about whether they have past experiences with teamwork. The students answered this question with either “Not at all”, “Once or Twice”, “Several Times”, or “Many Times”.
[BT_PastPositive] Students rated themselves on a Likert 5-point scale to this statement, ranging from “Strongly Disagree” to “Strongly Agree,” to the statement, “My previous teamwork experiences were generally positive.” A student’s past teamwork experience is considered to be positive if he or she scores this item as “agree” or “strongly agree”.
This paper aims to study students’ self-rated “Extraversion” (tendency of students being quiet or speaking up in a group) and to understand what might affect students’ perception that they will be quiet in a team. Thus, “SpeakUp” and “BT_BelongingConcern” are chosen as predictors since they have the highest correlation coefficient with “Extraversion” as observed in Table 1. “BT_Past-Groups” and “BT_PastPositive” are used to separate the data sets into smaller sets, where the separation method is explained above.
To answer the research questions, I first performed data visualization to see if there are any interesting trends among the data. As the data are discrete, most of the points ended up overlapping one another, thus a density plot in the form of a heat map was chosen so that the trends can be observed clearly as in Figure 6 and 7. In these plots, all 2082 students’ responses were used as I wanted to see how students rated themselves in terms of “Extraversion”, “BT_BelongingConcern”, and “SpeakUp”. To test the first hypotheses, the correlations, τb, between “Extraversion” and “BT_BelongingConcern” and “Extraversion” and “SpeakUp” are calculated. Since this study focuses on the analysis of ordinal data, I will be using the Stuart-Kendall Tau-b to calculate the correlation coefficient rather than Pearson’s r (Khamis 159). The correlation is calculated using R’s KendallTauB function.
After looking at the mean score of “SpeakUp”, “BT_BelongingConcern”, and “Extraversion”, I was curious to find the population mean for these variables. In order to approximate the means, I used the Bootstrap method. The 95% confidence interval was calculated using R’s in-built boot.ci() function. As the sample size was large and the three variables were approximately normally distributed, I used the normal confidence interval of the function. The normal confidence interval can be expressed as the following:
given that are the Bootstrap samples
is the standard error for the Bootstrap samples
is the and quantile of standard normal distribution
After obtaining the basic information about the three variables to be tested, bootstrap is once again used to test the third hypothesis by bootstrapping 10000 times (B=10000). The data was separated out into four smaller datasets, which were data of students with many teamwork experiences (6 or more times) and students with less or no teamwork experience (less than 6 times); and data of students with past negative experiences (score of 3 or less) and students with past positive teamwork experience (score of 4 or more). For hypothesis 2a, the difference in means for the “SpeakUp”, “BT_BelongingConcern”, and “Extraversion” variables were calculated through:
where M means many while L means less.
For hypothesis 2b, the difference in means for the “SpeakUp”, “BT_Belongingness”, and “Extraversion” variables are calculated through:
where P means positive while N means negative. For each of the differences in means, their respective confidence intervals are calculated using the same normal boot.ci() function.
Moving on, assuming that for some reasons, the instructors were unable to access the students’ Extraversion scores and had to predict students’ rate of speaking in groups based on other variables collected in the survey form, then it is essential to figure out what variable is the best predictor in determining a student’s Extraversion score. In order to achieve this, Leave-One-Out Cross-Validation can be used. In this study, a cost of one would be paid if quiet students are classified as talkative and vice-versa. The cross validation implementation uses 10000 replications to determine which of the two variables has the lowest average loss.
In addition, hierarchical clustering is used to cluster the students so that when new points (future students’ responses) are obtained, instructors will have a better understanding of which clusters the students belong to, easing lecturers in the process of assigning the students into teams.
Before I start testing the hypotheses, performing prediction or clustering the data, I first computed the correlation between each variable and the “Extraversion” variable to determine the best two predictors to be used. Thus, according to Table 1, we see that the top two predictors are “SpeakUp” and “BT_Belongingness”. Do note that the correlation coefficient value in Table 5 is the absolute value of the original value.
|| cor.val |||Predictors|
After finding the variables that have the highest correlations with “Extraversion”, the sample mean, bootstrapped mean confidence interval, bias and mean squared error (MSE) of “Extraversion”, “SpeakUp”, and “BT_Belongingness” are calculated as shown in Table 6.
|Variable||Mean||Bootstrap C.I. (95%)||Bias||MSE|
Hypothesis 1: There is a correlation between the metric about the rate of students speaking up in groups (Extraversion) and the metric about students’ beginning of term belonging concern score (BT_BelongingConcern) and the metric about whether the students will hold back ideas to ensure other group members stay happy (SpeakUp).
From Table 1, it is noticeable that the correlation between “Extraversion” and “SpeakUp” is not zero. Using the Stuart-Kendall Tau-b, the correlation between “Extraversion” and “SpeakUp” is around τb = 0.32 with a confidence interval of (0.3160, 0.3810). Therefore, since 0 is not included in the confidence interval, I will reject the null hypotheses, H0 in favor of the alternate hypotheses. This result actually agrees with the conclusions obtained in previous research where quiet students will often express agreement while talkative students are seen to be quite willing to disagree with other group members.
From Table 1, it is noticeable that the correlation between “Extraversion” and “BT_BelongingConcern” is not zero and by using the Stuart-Kendall Tau-b, the correlation between “Extraversion” and “BT_BelongingConcern” is around τb = –0.2217 with a confidence interval of (–0.2560, –0.1875). Therefore, since 0 is not included in the confidence interval, I will reject the null hypotheses, , in favor of the alternate hypotheses. This result also agrees with the conclusion from previous research that quiet students suffer from tensions between belonging and isolation when interacting with group members (Medaille and Usinger 254).
Hypothesis 2a: A student with many previous teamwork experiences is more likely to score high in “Extraversion”, high in “SpeakUp” and low in “BT_BelongingConcern”
From Table 3, it is observable that there is a difference in mean in the “Extraversion” and “BT_BelongingConcern” variable between students with many teamwork experiences and less teamwork experience. This is because the boot-strapped confidence intervals for these two variables do not include 0 in them. Students with many teamwork experiences score an average of almost 2 points higher in terms of extraversion, meaning that they self-rate as more often speaking in groups. Moreover, students with many teamwork experiences score almost 1.7 points less in terms of BT_BelongingConcern, meaning that they are more likely to expect to fit right into the course when the course has just started. I believe this is due to previous teamwork experiences giving students the confidence to express themselves in groups and reduce their fear towards teamwork projects in the new course. Although the difference in mean for the “SpeakUp” variable is about 0.8, I am unable to conclude that there’s actually a difference in “SpeakUp” variable between the two types of students due to the fact that the confidence interval contains 0. I believe that other factors, such as respecting others’ opinion or cultural constraints (Medaille and Usinger 243) might be more important than teamwork experiences.
|Variable||∆||Bootstrapped C.I. (95%)|
Thus, I will reject the null hypothesis in favor of the alternate hypothesis that there is indeed a difference in the mean score for “Extraversion” and “BT_BelongingConcern” among the two types of students. However, there is no clear evidence for me to reject the null hypothesis to conclude that there is a difference in mean score for the “SpeakUp” variable among the two types of students.
|Variable||∆||Bootstrapped C.I. (95%)|
Hypothesis 2b: A student with past positive teamwork experiences is more likely to score high in “Extraversion”, high in “SpeakUp” and low in “BT_BelongingConcern”
From Table 8, it is observable that there is a difference in mean in the “Extraversion” and “BT_BelongingConcern” variables between students with positive teamwork experiences and negative teamwork experiences. Surprisingly, the result from this test is similar to the ones obtained from Hypothesis 2a. The boot-strapped confidence intervals for “Extraversion” and “BT_Belongingness” do not include 0 in them. Students with past positive experiences score 1.7 points higher in terms of extraversion, meaning that they identify as often speaking in groups, and 1.7 points lesser in terms of BT_BelongingConcern, meaning that they expect to fit right into the course when the course has just started. I believe that past positive teamwork experiences have a similar effect as many teamwork experiences in terms of giving students the confidence to express themselves in groups and reduce their fear towards teamwork projects in the new course. Once again, I believe that Medaille and Usinger’s explanation can be applied here to explain why there is no clear mean difference in the “SpeakUp” variable between the two types of students (243). Thus, I will reject the null hypothesis in favor of the alternate hypothesis that there is indeed a difference in the mean score for “Extraversion” and “BT_BelongingConcern” among the two types of students. However, since the confidence interval does include 0 for the “SpeakUp” variable, there is no clear evidence for me to reject the null hypothesis that there is a difference in mean score for the “SpeakUp” variable among the two types of students.
|Variable||Average Loss||Average Cut-off point|
As mentioned in the introduction, assuming that the lecturers are unable to access the “Extraversion” score directly due to some reasons, then a variable with the least average loss must be chosen as the predictor to predict whether a student is expected to be quiet (Extraversion score is 4 or below) or talkative (Extraversion score is 5 or above). By computing the average loss for both of the variables with the highest correlation coefficient with “Extraversion”, it is observable that “BT_BelongingConcern” has a lesser average loss in predicting whether a student is quiet or not, even though the semantic relatedness with “SpeakUp” is high. Moreover, since the average cut-off point is 3.06, it means that students scoring a point of 4 or more in BT_BelongingConcern have a (1–0.3823)*100% = 61.77% likelihood of being a quiet student since “BT_Belongingness” and “Extraversion” are negatively correlated. However, if “SpeakUp” is used as a predictor, then only a student who scores a point of 7 out of 7 has a (1–0.4543708)*100% = 54.5629% accuracy of being a talkative student. Therefore, the result shows that “BT_BelongingConcern” is a better predictor in predicting whether a student is quiet or not. I believe that this statement makes sense as if a student feels that he or she does not fit into a class, then the student might choose to be quiet in it. (Medaille and Usinger 254)
Lastly, hierarchical clustering is used to plot the distribution of students’ responses. Figure 5 shows the dendrogram for the hierarchical clustering. From the dendrogram, I chose to separate the data out into two smaller clusters as shown in Figure 6. According to Table 10, it is noticeable that if new data points fall into the second cluster (on the left in Figure 6), the probability of students being quiet increase (69% of the new points will be quiet); if the points are in the first cluster (on the right in Figure 6), the probability of students being quiet decreases, (16% of the new points will be quiet).
From the results, it can be observed that there are correlations between “SpeakUp” and “Extraversion” and “BT_BelongingConcern” and “Extraversion”. This suggests that among the students who responded to the Beginning of Term survey, those students who identify as often holding back ideas or preferences to keep a group happy are also usually the students who would identify as tending to listen more than speak. Moreover, from the survey, I learnt that if students feel that they do not fit into the course even before the term has begun, these students are also likely to listen more than they speak in a team discussion. Nonetheless, even though the aforementioned correlations between variables exist, the correlation coefficient is not large enough to show a strong correlation between the variables. I suggest that in the future, the survey forms can include more questions on students’ background so that future researchers can determine if there are other variables other than “SpeakUp” and “BT_BelongingConcern” that are affecting a student’s “Extraversion” score.
From the second hypothesis, it is noticeable that among the students who responded to the survey, if a student has many teamwork experiences or has past positive teamwork experiences, then the student is expected to be more talkative and more likely to fit into a new course that contains teamwork projects. I believe that past experiences gave students the courage and confidence to express themselves in teams and they no longer feel scared to communicate with others in the team. Owing to this, they expect themselves to fit into the new course more easily than other students who had no experiences. Nonetheless, there is no significant evidence showing that students with more experiences will speak up about their ideas or preferences even if it disrupts the group. I believe that for some students, the thought of interrupting other students’ ideas is rude so they choose to hold back their ideas. This explanation is similar to Medaille and Usinger’s statement of “silence in teams is the result of personal, social, academic, cultural and contextual constraints” (243).
It is important to note several limitations of this study. First, the result of the survey might not be a good representation of the students themselves. This is because sometimes the students might not accurately categorize themselves. For example, a high achieving student might have imposter syndrome and thus feel that he is doing badly and does not deserve to be on the team; or a student thinks that he or she is actually talkative, but in reality, he or she is quiet. Therefore, it is recommended to use an End-of-Term survey that includes both the students’ evaluation of themselves and their peer evaluations on them.
Secondly, although all the respondents are students and all of their courses contain teamwork projects and discussion, the nature of the teamwork projects and discussions might not be the same across different disciplines. For example, a team discussion in humanities class might be interesting or relaxing while a team discussion in engineering class might be boring or stressful. As a result, students from different disciplines might have different attitudes towards the idea of teamwork and collaborative learning. Future research can be more precise by focusing on investigating whether the above results still hold in each discipline (Humanities, Social Sciences, Engineering, etc.)
Thirdly, the results and predictions obtained in this study are only applicable at the beginning of each course term. When the courses begin, there are even more factors throughout the semesters that may change a student’s attitude such as the quality of the lecturers, the course’s syllabus, and the quality of the peer discussions. It is suggested that future research can also focus on investigating how different factors that happen throughout the semester might change a student’s personality (such as from being quiet to talkative).
Lastly, it is to be noted that this survey focuses on students’ data collected during Fall 2020 and the result cannot be used to predict future students’ personality who enrolled in those classes unless a new survey form is filled out by the students and the same analytical method is performed. This result is useful in helping instructors to predict a student’s personality in the beginning of the semester, but it does not guarantee that the prediction is always accurate as the student’s personality changes throughout the semester. Therefore, an instructor should always observe any changes in students throughout the semester and make suitable changes to the group arrangement if necessary.
The author would like to thank Dr. Robin Fowler from the Technical Communication Program for the advice given throughout the writing of this paper, Dr. Cait Holman and the Center for Academic Innovation for data access, and Professor Mark Fredrick for the advice on the usage of statistical packages and formulae.
Avoiding Communication: Shyness, Reticence, and Communication Apprehension/.Beverly Hills:, c1984., .http://hdl.handle.net/2027/mdp.39015027243537
Beigpourian, B., “Using CATME to document and improve the effectiveness of teamwork in capstone courses.” 2019 ASEE Annual Conference and Exposition Proceedings. 2019.
Chin, Jeong Hin, “Predicting Team Project Score: It’s More about Team Harmony and Less about Individual Performance.” 2020 ASEE Virtual Annual Conference Content Access Proceedings, ASEE Conferences, 2020, p. 35075. DOI.org (Crossref), doi:.10.18260/1-2--35075
Espey, Molly. “Enhancing critical thinking using team-based learning.” Higher Education Research & Development 37.1 (2018): 15–29.
Gardner, Brenda S., and Korth, Sharon J.. “A Framework for Learning to Work in Teams.” Journal of Education for Business, vol. 74, no. 1, Routledge, Sept. 1998, pp. 28–33. Taylor and Francis+NEJM, doi:.10.1080/08832329809601657
Jin, Jun. “Students’ Silence and Identity in Small Group Interactions.” Educational Studies, vol. 43, no. 3, Routledge, May 2017, pp. 328–42. Taylor and Francis+NEJM, doi:.10.1080/03055698.2016.1277135
Khamis, Harry. “Measures of Association: How to Choose?” Journal of Diagnostic Medical Sonography 24.3 (2008): 155–62. Print.
Kim, Kyoungna, “Effects of active learning on enhancing student critical thinking in an undergraduate general science course.” Innovative Higher Education 38. 3 (2013): 223–235.
Lee, Hye-Jung, “Are High Achievers Successful in Collaborative Learning? An Explorative Study of College Students’ Learning Approaches in Team Project-Based Learning.” Innovations in Education and Teaching International, vol. 54, no. 5, Routledge, Sept. 2017, pp. 418–27. Taylor and Francis+NEJM, doi:.10.1080/14703297.2015.1105754
Lee, Hye-Jung, and Lim, Cheolil. “Peer Evaluation in Blended Team Project-Based Learning: What Do Students Find Important?” Journal of Educational Technology & Society, vol. 15, no. 4, International Forum of Educational Technology & Society, 2012, pp. 214–24.
Maneeratana, Kuntinee, and Sripakagorn, Angkee. Use of CATME for Teamwork Assessment in Engineering Projects. p. 5.
Medaille, Ann, and Usinger, Janet. “ ‘That’s Going to Be the Hardest Thing for Me’: Tensions Experienced by Quiet Students during Collaborative Learning Situations.” Educational Studies, vol. 46, no. 2, Routledge, Mar. 2020, pp. 240–57. Taylor and Francis+ NEJM, doi:.10.1080/03055698.2018.1555456
Pfaff, Elizabeth, and Huddleston, Patricia. “Does It Matter If I Hate Teamwork? What Impacts Student Attitudes toward Teamwork.” Journal of Marketing Education, vol. 25, no. 1, SAGE Publications Inc, Apr. 2003, pp. 37–45. SAGE Journals, doi:.10.1177/0273475302250571
Ruël, Gwenny Ch, Free-Riding and Team Performance in Project Education. p. 23.
Stuart, A. “The Estimation and Comparison of Strengths of Association in Contingency Tables.” Biometrika, vol. 40, no. 1/2, [Oxford University Press, Biometrika Trust], 1953, pp. 105–10. JSTOR, doi:.10.2307/2333101
Wilson, Laura, “Student Perceptions of Teamwork within Assessment Tasks in Undergraduate Science Degrees.” Assessment & Evaluation in Higher Education, vol. 43, no. 5, Routledge, July 2018, pp. 786–99. Taylor and Francis+NEJM, doi:.10.1080/02602938.2017.1409334