Introduction
The performing arts (e.g., circus, dance, music, theatre) excel at creating connection between the artist and audience and provide a unique social context in which individuals share in an aesthetic experience and affective response as part of a collective.1,2 The sense of “being moved” and social togetherness is heightened when the audience may engage with their body; for instance, the embodiment of music in synchronous swaying or clapping enhances enjoyment and promotes prosociality.1,3 The emergence of digital media, such as augmented reality (AR) and interactive technologies, offers performing artists new creative avenues to enter in relation with the audience and to mediate the collective experience. AR and virtual reality (VR) could enhance theatrical storytelling and allow the audience to participate in novel ways without replacing traditional theatre forms, but instead complement them by deepening emotional and social connections.4 Moreover, dramaturgical affordances of AR in transforming audience participation and interaction can reshape spectatorship and allow performances to become more immersive and responsive to real-time audience behavior.5 However, these novel formats challenge audience members’ traditional physical and conceptual relationship to the performers and one another, disrupting boundaries and de-hierarchising many theatrical norms in the process.5–9 The successful integration of immersive and interactive technologies in the performing arts thus necessitates a better understanding of how these technologies alter both the individual and collective aesthetic experiences, as well as the emotional engagement of the audience. The integration of physiological measures into the study of audience engagement provides distinct advantages over traditional observational or self-reported methods. These measures capture continuous, real-time, objective data that reflect emotional and attentional changes during a performance. Unlike post-performance surveys, which rely on memory and subjective interpretation, physiological data offer direct and unbiased insight into how the audience responds to specific moments.1,10–12 Furthermore, the use of biological measures capture real-time data in natural settings, providing neuroscience research with more ecologically valid insights into how the brain and body function in real-life scenarios. This timely information can assist creators of performance in optimizing their work as well as contribute to designing such performances that would dynamically adapt to the degree of engagement of the audience by highlighting the most captivating parts of the content.13 This information can also enable the creation of personalized experiences, such as in virtual reality, where performances can be tailored in real-time to an individual’s reactions, offering a new level of audience interaction by dynamically adjusting the content to enhance engagement and deliver more immersive, customized experiences. Rather than focusing solely on the mechanics of the technology, our aim is to understand how AR provides new opportunities for audience engagement on an individual and collective level, and, by extension, the social and emotional dynamics of participatory performances.
These questions were the impetus behind the participatory performance Lab-O, a creative collaboration between Montreal-based multi-media entertainment company Moment Factory and the circus company The 7 Fingers. Lab-O uses circus acts, motion tracking, augmented reality (AR) and narrator-guided group activities to orchestrate a social experiment with the goal of fostering a sense of connection. The show could be described as a new, hybrid format in the performing arts, where immersive technologies facilitate an interactive, participatory environment that still incorporates traditional performance element. It explored the theme of social cohesion and feeling connected to others. The success of this performance can be assessed by another technology: wearable sensors that allow for real-time recording of physiological states.10,12,14 Changes in autonomic nervous system (ANS) signals are informative in tracking participants’ emotional and attentional engagement throughout an immersive multi-sensorial activity,15 and can also elucidate the synchronous physiological activity that underlies social engagement, bonding and cohesion across the lifespan.16 Measures of physiological interpersonal synchrony can be an important complement to behavioural studies of social interactions and relational dynamics; their inclusion has advanced our understanding of the emotional bonds and sense of connection that arise in shared aesthetic or embodied experiences.1,15,17–19 Physiological synchrony has successfully been used to study the relationship between audience members and presenters.12 Lastly, studying the physiological reactions to a performance mediated by immersive technologies and with freely moving participants presents new practical challenges for data recording of an audience experience.
In this study, we investigated the audience’s experience during the prototype phase of Lab-O and addressed both the logistical and theoretical questions of real-time physiological signal recording in this context. We used wearable sensors to record participants’ ANS signals throughout the performance as a read-out of participants’ individual attentional states and as markers of moments of interpersonal synchrony. We present the findings of our proof-of-concept study and provide recommendations for investigating individual and collective experiences in emerging performance formats.
Methods
Participants
Ethical approval was obtained from the Institutional Review Board of McGill University (A05-B42–22B) and the National Circus School Review Board (CER 2122–13C). Participants were recruited through an email invitation sent to all show attendees several weeks in advance of the show containing details about the study. All individuals who volunteered by responding to the email were accepted to participate in the study. Participants arrived 30 minutes in advance of the show to be briefed on the study and signed an informed consent form. In total, 20 participants (55% female; aged 21–65 years old) took part in the study.
Description of activity
The participants engaged in a 50-minute participatory performance called Lab-O. The show took place in a large open room and used motion tracking and AR projections on the walls and floor to create immersive interfaces with which the participants interact throughout the performance. Before entering the space, participants were instructed to put on a silver tunic, underwent a mock screening test and were assigned a number, giving the sense that they were all participating in a playful psychosocial experiment. To further establish this scenario, a disembodied voice, the “master of ceremonies,” provided instructions to participants throughout.
The show was broken down into eight distinct activities, each with their own AR interface and accompanying audio (descriptions in Table 1 and photos in Figure 1). A guide helped participants discover the various interfaces in each section, and a few circus artists were planted amidst the participants to perform at various moments. The integration of circus performers, who were initially mixed in with the group, was intended to demonstrate the creative potential contained within each of us, while also challenging perceptions and encouraging openness to seeing beyond our assumptions. The addition of circus performances enhanced the experience for participants, offering them a spectacle as well as making them an integral part of the experience. The aim was also to encourage the audience to join in with the artists and play with this new-genre playground. This hybrid experience blended elements of traditional performance with participatory elements, combining immersive technologies, audience interaction and artistic performance. Rather than relying solely on performers to deliver the artistic content, this work involves both audience participation and performative moments, creating an integrated experience that blurs the line between observer and participant. The hybrid nature of this project lies in its focus on the interactive engagement of participants, while still incorporating performative elements where professional artists take on significant roles within key segments.
Titles, descriptions and artistic intention of Lab-O show segments.
Section title |
Description of the activity |
Artistic intention |
|---|---|---|
Move Together |
A subset of participants entered the room early and participated in a movement activity by interacting with instructive symbols displayed on the floor. The remaining participants entered and spread around the room, watching the early group dance until everyone had settled. |
The goal was to encourage the audience to move (dance) through simple interactions with the system. By following straightforward movement instructions, the creators aimed to break down the mental barriers and inhibitions people often feel when asked to “dance”: self-consciousness, self-censorship, judgment, etc. The audience followed a learning curve, progressing from simple to more complex movements. They moved from being aware of themselves in isolation to becoming conscious of the space around them and eventually to noticing the presence of others (with the rest of the group joining without even realizing it). There were no performers in this section. |
Survey |
After a word of welcome, participants were asked to answer a series of random questions projected on a large screen. Participants responded by moving to one of four answers projected on the floor (e.g., “Never,” “Sometimes,” “Often,” “Always”) that best represented their experience. Once all participants had placed themselves, the resulting percentages were calculated and displayed on the screen. Certain questions (e.g., |
This exercise began to foster interaction between participants, building a sense of belonging to a group through a fun activity. By engaging them physically and asking simple questions, participants could identify shared characteristics, quirks and a sense of connection with others. There were no performers in this section. After a word of welcome, participants were asked to answer a series of random questions projected on a large screen. Participants responded by moving to one of four answers projected on the floor (e.g., “Never,” “Sometimes,” “Often,” “Always”) that best represented their experience. Once all participants had placed themselves, the resulting percentages were calculated and displayed on the screen. Certain questions (e.g., |
Circle |
Participants gathered in a circle with the guide in the centre. The guide led a series of synchronized movements, including clapping, rubbing the hands together, snapping fingers and clapping on knees with accompanying sound effects. Near the end of this activity, participants performed a coordinated arm wave around the circle. |
The guide (performer) established a basic language with the group, demonstrating that communication could occur without filters (and without technology), human to human, using simple signs and movements. The performer led the participants to synchronize with each other and the group, introducing them to the concept of timing. Participants became part of a sequence of movements waiting for their turn and collaborating with the group toward a shared goal. The intent was for participants to recognize they were part of something larger than themselves where each person had a specific time and role. |
Musical Harmony |
Participants created a musical composition by stepping on different squares projected on the floor, which each square represented a note. The order in which participants place themselves determined how the piece was recorded. Participants were invited in groups to record a segment, and then the final composition was played back. This happened twice to give participants an opportunity to familiarize themselves with the interface. At the end of Musical Harmony, participants gathered in a circle and two planted performers were called in. At first, they follow the narrator’s instructions, slowly moving together, but eventually progress into a hand-to-hand acrobatic performance. |
Musical Harmony was created by generating live music from the participants’ actions, giving them the power to create sound. They could hear the direct consequences of where they chose to stand and how they decided to move. Each musical piece was unique and directly connected to the participants’ actions. At the end of this section, two performers entered the center, initially behaving like participants, and gradually transformed into a full-fledged performance. This transition symbolized that anyone could become the center of attention and take an active role in the experience. The performers demonstrated additional possibilities for musical and sound variations through their movements. In the hand-to-hand performance, the musical tempo shifted based on the height of the act (slower on the floor, faster in the air). This high-skill performance was both impressive and engaging, demonstrating grace and connection between the performers. |
Avoid the Red |
At the beginning of this section, the narrator gave participants one command: “avoid the red”. A song began to play as a projected multi-coloured pattern scrolled from the wall down onto the floor with red segments that participants had to avoid by running around or jumping over. Their movement was tracked and the group’s score was projected at the end of the activity. The song and scrolling projection then ran a second time, significantly faster than the first. |
In this sequence, the intention was to make the participants move, raise their heart rate and increase their body temperature by engaging them in a more visceral game. They were encouraged to fully engage their bodies, become aware of those around them and manage their movements relative to others. This sequence built to a climax of activity and emotion, with a comedic release as the game escalated to an impossibly challenging level. There were no performers in this section. |
Water Break |
A short (5 minute) break where participants were invited to catch their breath and drink some water provided by show assistants. |
After the previous intense section, participants were allowed to take a break and calm down before starting a new chapter. |
Painting |
The narrator instructed participants to attempt a synchronized jump. After two attempts (the second more successful than the first), a new interface was activated. Participants’ movement was traced as lines on the floor. The narrator called forth participants by their number, two or three at a time, to attempt to draw something together (a heart, square, arrow, sheep). This required them to coordinate their movement and adjust according to the image forming on the wall. Other participants watched and the activity concluded with everyone drawing a spiral together. |
This section was a progressive buildup toward collaboration. A learning curve was created, starting with a simple task for one person, moving to collaboration between two participants (which generally worked well) and finally to a more complex task involving four participants (which typically failed unless one person took the lead). The focus was on observing how participants collaborated and organized with their partners to complete the task. There were no performers in this section. |
The rest of the group observed and waited to see if they would be called to the center. Did they feel anticipation or anxiety? Were they relieved or disappointed if not chosen? Did they root for their companions in the center? Were they excited when their peers succeeded? |
||
Rorschach |
This section followed the same drawing principle as the last, but participants’ movement were captured as an abstract painting on the floor. Participants were invited to jump on “power buttons” on the floor, which changed the colors and shapes generated. Participants moved around to paint an abstract image, which was then captured and projected on the wall. At this point, the narrator briefly introduced who Rorschach was and invited a participant to say what they saw in the image. The group painting and Rorschach analysis was then repeated a second time. |
After learning the drawing rules from previous sequences (whether by observing or participating), everyone had the chance to engage in the activity, now with an additional parameter: height was associated with a color. This created a collective visual artwork, with each participant contributing to the overall picture. The power buttons allowed participants to reset the canvas (which filled up quickly due to the number of participants) and provided more control to those who could reset the game for everyone. The Rorschach interpretation was a nod to the scientific nature of the experience, evoking the exploration of the subconscious and highlighting the similarities and peculiarities within the group. |
Cyr Wheel |
The narrator once again gave the “avoid the red” command, but this time the red projection was around a performer entering the space with a Cyr wheel, causing participants to clear space for the performance. At the conclusion of his act, the performer held the Cyr wheel in place to act as a gate, which participants were invited to walk through to close the show. |
The Cyr Wheel performer displayed impressive acrobatic skills and spatial awareness to avoid collisions with the audience. This act showcased the breadth of movement and exploration of space that one can achieve. Participants used previously learned cues (e.g., avoiding the red) and moved around the performer. The guides encouraged participants to crouch or stand, creating a collective choreography around the performer. Participants were aware of their own bodies, the performer’s presence, the people around them and their shared environment. |
Photos of A) Move Together, B) Musical Harmony, C) Rorschach and D) Cyr Wheel segments of Lab-O. Photo by Kun Chang.
Physiological measures
The ANS has two sub-systems: the parasympathetic nervous system (associated with “rest and digest”) and the sympathetic nervous system (associated with “fight or flight”). When humans experience stress or anxiety, the sympathetic system gets excited and the parasympathetic system is inhibited, while the opposite occurs in relaxed states. Thus, changes within ANS signals such as electrodermal activity, heart rate and skin temperature (Table 2) relate to internal emotional or mental states.15
Physiological variable studied.
Physiological variable |
Description |
|---|---|
Electrodermal activity (EDA) |
Electrodermal activity (EDA), also known as galvanic skin resistance and skin conductance, reflects the amount of sweat in the skin, and can change spontaneously or reflexively in response to both external and internal stimuli. These stimuli include emotional thoughts, memory recall or encounters with novel, startling or threatening situations and may evoke electrodermal reactions (EDRs) that appear as transient peaks in the signal.15 |
Fingertip skin temperature (TEMP) |
Fingertip temperature varies as a function of homeostatic thermoregulatory vasodilation and vasoconstriction of blood vessels.20 It is used as a read-out of sympathetic activity but is dependent on the subject’s body temperature. Sympathetic excitement causes vasoconstriction, reducing heat radiation when the initial temperature is above 33.2°C. Below this temperature, vasodilation occurs and vasoconstriction is not possible. Fingertip temperature has been shown to increase in response to mental relaxation and to decrease in response to stimuli such as sudden noises, fear, pain and mental stimulation.15 |
Heart rate variability (HRV) |
The heart rate variability is calculated from the blood volume pulse (BVP). When individuals concentrate on the external environment and engage in heightened sensory information gathering, their heart rate tends to decrease. Conversely, during internal focus or when sensory input is reduced, heart rate has been found to increase.20 Additionally, higher heart rates have been observed during elevated levels of stress and anxiety.12,16 Changes in the inter-beat intervals over time are reflected in the HRV, with a steady heart rate corresponding to a low HRV and a variable heart rate corresponding to a high HRV. |
Data collection
Before entering the show, participants were briefed on the research study and the physiological monitoring technology. Participants were given a triple-physiology sensor (TPS; Thought Technology Ltd, Montreal, QC, Canada), a wearable device that is attached to the fingertip by a Velcro strap. The TPS records three ANS signals: 1) EDA, 2) TEMP 3) and BVP, which is used to obtain the heart rate and HRV. These signals are sampled at 15, 15 and 75 Hz, respectively, and are transmitted wirelessly using Bluetooth to an application on an Android phone.15,20
Participants were instructed to attach the TPS device on their non-dominant index finger and all devices were tested for proper functioning. Baseline physiological measures were obtained for a period of two to five minutes in a waiting room outside of the show room. The research team entered the performance space with the participants and set up behind a screen to monitor the Bluetooth connections between the TPS devices and the phones. We used timestamped comments captured within the signal recording application to mark the beginning of the show and the transitions between sections and for additional observations throughout the show. We subsequently synchronized the signals across phones by marking the show start timestamp, enabling us to compare the physiological recordings across all participants.
Data analysis
Eight participants were excluded from the analysis: two before any pre-processing and the remaining six due to partially missing or erroneous data to ensure a consistent number of datapoints throughout all analyses. The “Move Together” and “Cyr Wheel” sections were excluded entirely from analysis due to insufficient data, as the former involved only a subset of participants and the latter occurred at the end of the show when many TPS sensors had lost stable Bluetooth connection.
The data from the remaining twelve participants underwent pre-processing to eliminate non-physiological artifacts. We applied a 1-D median filter (order n = 75) and a moving average filter with a 0.75-second window to smooth the data. Additionally, we utilized a one-Euro filter on the EDA signal to reduce jitter and lag, an exponential decay filter to the TEMP signal and a cubic smoothing spline to the HRV signal.
Audience engagement
In this context, audience engagement was measured via physiological arousal. Following the pre-processing, the signals were divided by performance section and analyzed individually, extracting the following features over a sixty-second window: 1) average EDA slope, 2) average TEMP slope and 3) average HRV. These features were utilized in the following manner: 1) the standard deviation of the slopes of the EDA was used to determine the presence or absence of EDRs; 2) the median of the slopes of fingertip temperature was used to determine its general increasing/decreasing trends; 3) HRV was used to note moments of higher or lower variation throughout.
Interpersonal physiological synchrony
We used the single-session index (SSI) metric to quantify group synchrony in the EDA signal across the various show segments. SSI was calculated for all possible dyadic combinations between participants, and the mean of the resulting SSIs was calculated to represent the group SSI.
The EDA signals from both participants in the dyad were standardized using Z-scores to enhance their comparability.21,22 The average slopes of the EDA signals were then determined by analyzing overlapping five-second segments with a step interval of one second.15 To evaluate physiological alignment between the participants, referred to as physiological concordance (PC), successive Pearson correlations were computed between the two sets of trends, employing a sliding window of fifteen seconds and a step interval of one second. The SSI was computed from a correlation array R by applying the natural logarithm to the ratio of the cumulative positive correlations and the magnitude of the cumulative negative correlations:
The SSI was computed over the length of each show section to analyze dyadic interpersonal physiological synchrony amongst participants during the performance.23 To assess group synchrony, the mean SSI was calculated for every dyad for each performance section, providing a coarse measure of which sections evoked the most synchronization. The statistical significance of the mean SSI was evaluated using Monte Carlo shuffling, a technique also used in earlier investigations of the PC index.24 For each pair under consideration, the Pearson correlation was calculated using the EDA signals, with the time sequence of one participant kept intact while the time series of the other participant was randomly shuffled. This process was iterated 1,000 times. The actual SSI measurements were then contrasted with the distribution of shuffled SSI values. Results were deemed statistically significant if they surpassed the upper fifth percentile of this shuffled distribution and otherwise set to zero.
Statistical analysis
Data was tested against the normal distribution using the Kolmogorov-Smirnov test and the Shapiro-Wilk test, both strongly rejecting the null hypothesis of normality. Accordingly, Friedman’s non-parametric test was performed in place of the repeated-measures ANOVA to test for statistically significant differences between the show segments within each of the three features. Following a Bonferroni-adjusted result of p<0.05, pairwise post-hoc comparisons were performed using a paired-sample Wilcoxon signed-rank test. For every feature and every pair of show segments, the significance of the difference was assessed against a threshold of 0.05, Bonferroni-adjusted for the total number of pairs. When significant, individual p-values were reported along the corresponding pair.
Results
Audience engagement
The pre-processed physiological signals presented in Figure 2 correspond to the activities of the performance described in Table 1. Peaks in the EDA signals during “Avoid the Red” and “Rorschach,” activities involving high levels of physical interaction and/or unexpected elements, indicate high levels of arousal or engagement. TEMP variations were observed throughout the performance with many moments of inflection corresponding with a change in activity. Decreases in skin temperature were noted particularly during the “Avoid the Red” or “Circle Part” segments, indicating sympathetic nervous system activation in response to changing external stimuli. Conversely, periods of relaxation, such as “Musical Harmony” or the water break, showed a gradual increase in fingertip temperature. Large peaks in HRV were observed during the “Musical Harmony” segment.
Group average (n=12) of audience EDA (electrodermal activity), TEMP (fingertip skin temperature) and HRV (heart rate variability) during the entire show over time with timestamps of the activities described in Table 1.
The mean standard deviation of the EDA slopes, the mean of the median TEMP slopes and the mean average HRV across all show activity segments are illustrated in Figure 3. The standard deviations of EDA slopes were highest in the “Musical Harmony” segment, suggesting greater fluctuation in physiological arousal levels, although this variation was only significantly different from that observed in the “Circle” segment (p<0.05). The variation in TEMP differed significantly across segments (p<0.05), with the most stable TEMP observed during “Musical Harmony”. The largest rates of increase in TEMP were observed in “Circle” and “Painting,” suggesting periods of reduced physiological arousal and increased relaxation. While variations in heart rate were observed throughout, they were particularly elevated in the “Circle” segment and differed significantly between “Avoid the Red” and “Painting” (p<0.05).
Mean standard deviation of the EDA slopes (top), mean of the median TEMP slopes (middle) and mean average HRV (bottom) across all show segments. Bonferroni-adjusted p-values of significant pairs are indicated.
Physiological synchrony
The SSI for the EDA signals in Figure 4 illustrates the group mean within each show segment, calculated from all possible participant pair combinations and compared to chance (Monte Carlo shuffling). A representative example of synchronized EDA signals between two participants is shown in Figure 5. Mean SSI values were higher in all show activities compared to during the “Water Break,” which is represented as a control line given that we do not expect behavioural or physiological synchrony in this segment. Positive mean SSI values suggest high group physiological synchrony during the “Survey” and “Avoid the Red” segments, whereas participants’ EDA signals are less synchronized in “Rorschach”. However, a comparison by segments did not reach statistical significance (p=0.35), likely due to the high variability in the data; thus, there is not enough evidence to reject the null hypothesis.
Mean single-session index (SSI) for the EDA signals for each show segment computed for all possible dyads among the participants with “Water Break” illustrated as a baseline.
Representative example of EDA signal synchrony between two participants (P15 and P19) during the “Musical Harmony” segment.
Discussion
This study aimed to test the feasibility of real-time physiological signal recording as a measure of audience engagement in a novel immersive performance, and to investigate whether physiological synchrony across audience members would vary based on their level of participation and interaction. We successfully recorded physiological signals from 20 participants while they actively engaged in a participatory performance, moving in a large space in response to an AR environment. Following data quality control, we were able to analyse continuous physiological signals from twelve participants, who showed distinct responses in EDA, TEMP and HRV corresponding to the different activities through the show. Moreover, we observed trends in group physiological synchrony, as indicated by positive SSI values. These outcomes underscore the potential of using wearable sensors for monitoring physiological reactions during immersive and interactive performances and provide preliminary findings on the individual and interpersonal experience in emerging artistic contexts.
The analysis of EDA, TEMP and HRV revealed distinct physiological responses associated with different segments of the performance. Activities that required physical engagement and sustained attention, or that involved unexpected elements, such as “Avoid the Red” and “Rorschach,” elicited significant peaks in EDA. These segments involved coordinated activities and collective participation, fostering a sense of unity and shared experience among the audience members. This aligns with previous research showing that interactive and physically engaging activities can enhance emotional engagement.1,14,19 Conversely, segments like “Painting,” which involved more passive participation, were associated with increasing TEMP, suggesting states of relaxation and lower arousal. These findings support the idea that both active and passive elements of a performance can contribute to overall engagement by providing a balance of excitement and relaxation.3,16
The higher group SSI values during “Survey” and “Avoid the Red” segments suggest that activities fostering interaction and collective movement can enhance synchrony among participants. In these two segments, we can assume that the fact that the instructions were well-defined and delimited in time meant that the group’s physiological reactions were more synchronized overall. These results expand on previous research which showed that immersive environments facilitate a sense of shared presence among audience member.8 While both activities focused on an individual task behaviour rather than interpersonal interaction, previous research suggests that co-presence alone can contribute to emotional and physiological alignment. For example, spectators of live theatrical performances exhibited increased cardiac synchrony and a subsequent convergence of emotional evaluations.1 In the case of the “Survey” segment, individual action may have additionally been reinforced by observing the response selection of other participants. While the analysis of physiological synchrony (SSI) across audience members did not reach statistical significance, the trends observed offer potential insights into how immersive and participatory performances may foster a shared emotional and attentional response. Although these results should be interpreted cautiously due to the lack of statistical significance, they provide a basis for further investigation into how shared physiological responses align with emotional and cognitive engagement and underlie joint attention and action in interactive performances. This preliminary insight adds a new dimension to our understanding of how digital and immersive technologies could enhance not only individual but also collective audience experiences in future studies. Given a larger sample size, it may be interesting to compare the physiological synchrony among participants who moved to the same response circle on the floor with those who moved to another.
While not possible given our sample size, sub-group analyses may have offered a more granular perspective on physiological synchrony across specific audience members. While “Survey” and “Avoid the Red” involved the same instructions for all participants, several of the other show segments divided participants into sub-groups. For example, in “Painting,” only one to four participants digitally “painted” on the floor with their movement, while the rest observed. This likely contributed to the high variability in individual responses, thereby diluting group effects. With a greater number of participants, we may have uncovered differences between those actively engaged versus witnessing an activity. This would have been particularly interesting in “Musical Harmony,” in which the group was split in two to compose a piece of music. Given that physiological (heart rate, respiration rate, skin conductance response) and movement synchrony has been observed amongst audience members during classical music concerts, which was linked to self-reported emotional and aesthetic experiences,3 it is compelling to consider how controlling the music through bodily movement and collective action in Lab-O would impact these measures. Lastly, given individual variability in physiological responsiveness, including biological sex differences,25 future research with larger sample sizes would benefit from separating participants into sub-groups based on their primary responsive ANS signal as determined in a baseline recording. Overall, this highlights the complexity of measuring group synchrony in a performing arts context and suggests that individual differences in engagement and interaction levels can influence overall synchrony.12,26
The results of this study need to be considered in light of several limitations. First, the recruitment strategy relied on an email invitation sent to individuals who had signed up for the show. This approach may have led to a participant group who were not entirely representative of a normal audience. We recognize that the immersive nature of the performance could introduce barriers to accessibility for diverse audience members, an important consideration for future research.27 In future studies, recruitment efforts should aim to reach a more diverse audience in terms of demographic variables and prior exposure to immersive experiences to better understand how different groups respond to such performances. Secondly, it is possible that the level of physical activity required during certain parts of the performance affected the physiological variables recorded. Engaging in physical activity typically leads to an increase in EDA and TEMP, while HRV tends to decrease. There may also be a carry-over effect between sections, where the physiological responses from one section persisted into the next, thereby influencing the measured variables.27 However, most sections involved low to medium intensity, with participants primarily doing brief walking or arm movements while remaining in place. The only higher-intensity segments were “Avoid the Red” and “Cyr Wheel,” the former was followed by a break, and the latter marked the end of the performance. While this could be seen as a limitation of our study, we feel that these factors do not undermine the exploratory nature of our research, particularly given that we observed changes in physiological states that did not map onto the physical intensity of the corresponding activity. To address these effects in future studies, we suggest working with the artistic directors early in the creation to incorporate recovery periods between segments within the performance story arch.
Considerations and recommendations
Conducting research in a performance context presents several practical challenges, and incorporating clear objectives and a research protocol into an artistic experience requires a careful approach. Firstly, the dynamic and unpredictable nature of live performances can complicate the standardization of data collection methods; it necessitates flexible and adaptive research designs and requires collaboration between researchers and artists to develop methodologies that are unobtrusive yet effective. This issue is highlighted in a study on audience biometric and self-report data by Latulipe et al.,11 which reported that while performing arts experts were interested in the data, they were cautious in its interpretation and weary of its infringement on creative expression. We felt a similar tension in our work; our requirements for data collection and scientific rigour were at times at odds with the emotional journey and aesthetic experience the creators wanted to prioritize. As Sedgman28,29 has argued, creative expression and scientific research need not be mutually exclusive, and performance-based research can incorporate audience feedback in ways that enhance rather than disrupt the artistic flow. Furthermore, audiences experience a sense of “realness” that is closely tied to the emotional and affective dimensions of how they take part in immersive shows.30 This suggests that capturing both the emotional impact of the performance and the physiological data can provide a more comprehensive understanding of how audiences engage with immersive, participatory shows. To illustrate this with concrete examples and to guide future performance research endeavours, we outline three tensions we faced below:
Collecting self-report (questionnaire) data: While our initial intent was to gather both implicit (physiological) and explicit (self-report) data on audience experience, and therefore designed survey questions to include in a pre- and post- questionnaire, we ultimately had to discard this as a reliable source of data. As the creators wanted to set a certain ambience before and after the show, it was impossible to integrate psychometric scales or long-answer questions without disrupting the audience’s positive experience and immersiveness in the event. Moreover, given this was the prototype phase of Lab-O, the creators wanted to prioritize collecting feedback that would serve to improve the performance and AR interfaces, rather than that which would directly serve the research objectives. Lastly, the playful nature of the show, presented as a social experiment, contributed to many participants not taking the surveys seriously. Gallagher et al. suggested that surveys and interviews can be integrated into performances in ways that match the theme without reducing the immersive feel or emotional impact of the event.31 The survey activity at the beginning of the Lab-O show would have provided a perfect opportunity to integrate self-report measure within the performance’s storyline; unfortunately, the research collaboration emerged too late within the artistic direction to allow for it in this first iteration of the performance.
Tracking participants in the performance space: As the performance was filmed from above, tracking the show participants by way of a visual cue would have enabled us to note moments of behavioural interaction, which would in turn allow for a more precise analysis of dyadic and group synchrony. However, singling out participants in such a manner was directly in contrast to the group-feel the creators wished to establish, notably in giving all participants the same silver tunic to wear. Research demonstrates that the way participants engage with and reflect on their role in a performance can influence the group dynamics and sense of collective immersion.32
Wearable sensors and performance activities: While participants were instructed on how to wear the sensors and ensure they remained turned on throughout the performance, certain activities compromised our physiological data. For example, when participants were instructed to clap their hands or slap their knees, this caused artefacts in the EDA recording, which is highly sensitive to changes in pressure on the finger. Having a better understanding of all interactive activities in advance may have better prepared us to mitigate the noise in our data, for example by adjusting the tightness or placement of the sensors. Audience research can benefit from collaboration between creators and researchers to ensure that wearable technologies and other tools do not interfere with the artistic intent while still providing reliable data.31
These three examples highlight not only the need for continuous communication between the research and artistic team, but also the necessity of incorporating the research questions and protocol into the dramaturgy and artistic choices early on. While we strived towards such a process in the present collaboration, compromises are an unavoidable and important aspect of conducting research in naturalistic settings. A more seamless integration of real-time physiological recording in performances can not only elucidate how the incorporation of new technologies shape individual and collective audience experiences, but also can open new creative avenues.13 Artists can design interactive experiences that adapt in real-time to the audience’s emotional or attentional states, such as in changing pace, lighting or music, thereby creating a more immersive and personalized experience. We hope our preliminary findings, and the lessons learned in this proof-of-concept study, may inspire both research and creative ideas for future collaborations.
Conclusion
Overall, this study demonstrates the feasibility of using wearable sensors to measure physiological activity of freely moving audience members in a participatory performance, and reports on distinct physiological responses corresponding to changing attentional and emotional states throughout the proposed activities. Combining physiological recording with the new possibilities offered by immersive and interactive technologies may allow artists to better craft the emotional and immersive aspects of their performances, creating adaptive performance formats that consider both the individual and collective experience.
Acknowledgements
The authors received financial support from Subventions de partenariat en recherche appliquée et en technologie (PRAT), from Conseil de recherches en sciences naturelles et en génie du Canada (CRSNG) and from MITACS Accelerate.
Conflict of interest
None declared.
Acknowledgements
The authors would like to thank the companies the 7 Fingers and Moment Factory.
References
1 Ardizzi M, Calbi M, Tavaglione S, Umiltà MA, Gallese V. Audience spontaneous entrainment during the collective enjoyment of live performances: physiological and behavioral measurements. Scientific reports. 2020; 10(1):3813. doi:https://doi.org/10.1038/s41598-020-60832-7.https://doi.org/10.1038/s41598-020-60832-7
2 Corness G, Carlson K, Schiphorst T. Audience empathy: a phenomenological method for mediated performance. Proceedings of the 8th ACM conference on Creativity and Cognition. 2011 Nov 3–6; Atlanta, GA. 127–136 p.
3 Tschacher W, Greenwood S, Ramakrishnan S, Tröndle M, Wald-Fuhrmann M, Seibert C, Weining C, Meier D. Audience synchronies in live concerts illustrate the embodiment of music experience. Scientific Reports. 2023; 13(1):14843. doi:https://doi.org/10.1038/s41598-023-41960-2.https://doi.org/10.1038/s41598-023-41960-2
4 Pike S. Virtually relevant: AR/VR and the theatre. Fusion Journal. 2020;( 17):120–128.
5 Hunter E. Augmented Reality and Theatre. Theatre Journal. 2024; 76(2):177–195.
6 Webster J, Ho H. Audience engagement in multimedia presentations. ACM SIGMIS Database: the DATABASE for Advances in Information Systems. 1997; 28(2):63–77. doi:https://doi.org/10.1145/264701.264706.https://doi.org/10.1145/264701.264706
7 Li J, Gui X, Kou Y, Li Y. Live streaming as co-performance: Dynamics between center and periphery in theatrical engagement. Proceedings of the ACM on Human-Computer Interaction. 2019; 3(CSCW):1–22. doi:https://doi.org/10.1145/3359166.https://doi.org/10.1145/3359166
8 Weijdom J. Mixed Reality and the Theatre of the Future. Fresh Perspectives on Arts and New Technologies. 2017; 6:47–54.
9 MacArthur M, McLeod K, Mealey S. Distance, disruption, and de-hierarchisation: negotiating care in the virtual space of Zoom theatre. Research in Drama Education: The Journal of Applied Theatre and Performance. 2024; 29(1):24–42.
10 Wang C, Geelhoed EN, Stenton PP, Cesar P. Sensing a live audience. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2014 Apr 26-May 1; Toronto, ON, Canada. 1909–1912 p.
11 Latulipe C, Carroll EA, Lottridge D. Love, hate, arousal and engagement: exploring audience responses to performing arts. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2011 May 7–12; Vancouver, BC, Canada. 1845–1854 p.
12 Gashi S, Di Lascio E, Santini S. Using unobtrusive wearable sensors to measure the physiological synchrony between presenters and audience members. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2019; 3(1):1–19. doi:https://doi.org/10.1145/3314400.https://doi.org/10.1145/3314400
13 Muller L, Turner G, Khut G, Edmonds E. Creating affective visualisations for a physiologically interactive artwork. Tenth International Conference on Information Visualisation (IV’06). 2006 Jul 5–7; London, England. 651–657 p.
14 Han J, Chernyshov G, Sugawa M, Zheng D, Hynds D, Furukawa T, Padovani Macieira M, Marky K, Minamizawa K, Ward JA. Linking audience physiology to choreography. ACM Transactions on Computer-Human Interaction. 2023; 30(1):1–32. doi:https://doi.org/10.1145/3557887.https://doi.org/10.1145/3557887
15 Fu D, Serra NI, Mansion H, Mansion ET, Blain-Moraes S. Assessing the effects of nature on physiological states using wearable technologies. International Journal of Environmental Research and Public Health. 2022; 19(3):1231. doi:https://doi.org/10.3390/ijerph19031231.https://doi.org/10.3390/ijerph19031231
16 Mayo O, Lavidor M, Gordon I. Interpersonal autonomic nervous system synchrony and its association to relationship and performance–a systematic review and meta-analysis. Physiology & Behavior. 2021; 235:113391. doi:https://doi.org/10.1016/j.physbeh.2021.113391.https://doi.org/10.1016/j.physbeh.2021.113391
17 Dotov D, Bosnyak D, Trainor LJ. Collective music listening: Movement energy is enhanced by groove and visual social cues. Quarterly Journal of Experimental Psychology. 2021; 74(6):1037–1053. doi:https://doi.org/10.1177/1747021821991793.https://doi.org/10.1177/1747021821991793
18 Baranowski-Pinto G, Profeta V, Newson M, Whitehouse H, Xygalatas D. Being in a crowd bonds people via physiological synchrony. Scientific reports. 2022; 12(1):613. doi:https://doi.org/10.1038/s41598-021-04548-2.https://doi.org/10.1038/s41598-021-04548-2
19 Chabin T, Gabriel D, Comte A, Haffen E, Moulin T, Pazart L. Interbrain emotional connection during music performances is driven by physical proximity and individual traits. Annals of the New York Academy of Sciences. 2022; 1508(1):178–195. doi:https://doi.org/10.1111/nyas.14711.https://doi.org/10.1111/nyas.14711
20 Blain S, Chau T, Mihailidis A. Peripheral autonomic signals as access pathways for individuals with severe disabilities: a literature appraisal. The Open Rehabilitation Journal. 2008; 1(1). doi:10.2174/187494370080101002710.2174/1874943700801010027
21 Ben-Shakhar G. Standardization within individuals: A simple method to neutralize individual differences in skin conductance. Psychophysiology. 1985; 22(3):292–299. doi:https://doi.org/10.1111/j.1469-8986.1985.tb01603.x.https://doi.org/10.1111/j.1469-8986.1985.tb01603.x
22 Haataja E, Malmberg J, Järvelä S. Monitoring in collaborative learning: Co-occurrence of observed behavior and physiological synchrony explored. Computers in Human Behavior. 2018; 87:337–347. doi:https://doi.org/10.1016/j.chb.2018.06.007.https://doi.org/10.1016/j.chb.2018.06.007
23 Slovák P, Tennent P, Reeves S, Fitzpatrick G. Exploring skin conductance synchronisation in everyday interactions. Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational; 2014. 511–520 p.
24 Karvonen A, Kykyri VL, Kaartinen J, Penttonen M, Seikkula J. Sympathetic nervous system synchrony in couple therapy. Journal of Marital and Family Therapy. 2016; 42(3):383–395. doi: https://doi.org/10.1111/jmft.12152.https://doi.org/10.1111/jmft.12152
25 Dearing C, Handa RJ, Myers B. Sex differences in autonomic responses to stress: implications for cardiometabolic physiology. American Journal of Physiology-Endocrinology and Metabolism. 2022; 323(3):E281-E289. doi:https://doi.org/10.1152/ajpendo.00058.2022.https://doi.org/10.1152/ajpendo.00058.2022
26 Mogan R, Fischer R, Bulbulia JA. To be in synchrony or not? A meta-analysis of synchrony’s effects on behavior, perception, cognition and affect. Journal of Experimental Social Psychology. 2017; 72:13–20. doi:https://doi.org/10.1016/j.jesp.2017.03.009.https://doi.org/10.1016/j.jesp.2017.03.009
27 Fung S, Jacobson K, Pike J. PXR2020: Re-seeing the possibilities of theatre in virtual, augmented, and mixed reality. University of Toronto Press; 2022.
28 Sedgman K. Uncharted Territories in ‘Empirical’ Audience Research. Performance Matters. 2019; 5(2):169–172.
29 Sedgman K. Audience experience in an anti-expert age: A survey of theatre audience research. Theatre Research International. 2017; 42(3):307–322.
30 Jacobson K. Real-ish: Audiences, Feeling, and the Production of Realness in Contemporary Performance. McGill-Queen’s Press-MQUP; 2023. ISBN: 022801641X.
31 Gallagher K, Valve L, Balt C. Building new publics: using agile, community-engaged, and applied theatre methodologies as social intervention in audience research. Research in Drama Education: The Journal of Applied Theatre and Performance. 2021; 26(2):318–334.
32 Mealey SR. Recollection of an Audience: Style, Experience, and Change in the Study of Theatre Spectatorship [dissertation]. University of Toronto (Canada); 2020.