Research brief
Takeaway: Spatial audio can make video meetings feel more like sharing the same space. Hearing each voice from where the speaker appears on screen made conversation seem more interactive and others' thoughts easier to understand. It works by restoring the sound cues people naturally use to tell who is speaking.
Why this matters
- What was known
- Video meeting conversation has long been reported as stilted, and the few studies of spatial audio in meetings used simulated meetings and measured comprehension and memory rather than presence and turn-taking.
The few studies that exist (e.g. [2, 27, 46] tend to use simulated meetings to control the sound, and also tend to investigate comprehension and memory more than presence and turn-taking.
(§1, p. 2) - What this paper adds
- A controlled within-subjects experiment comparing spatial and mono audio in live video meetings, measuring perceived turn-taking, social presence and group task performance, with gender as a moderator.
This study, though modest in scope and findings, provides some of the first modern direct evidence that spatial audio has a positive effect on people's perceptions of their ability to engage in video meetings.
(§5.1, p. 6) - Who it matters to
- Designers and engineers of video meeting tools and audio devices, researchers of video meeting fatigue, social presence and turn-taking, and organisations concerned with meeting inclusion.
Spatial audio could make meetings more inclusive not only for women struggling with participation, but also neurodiverse communities and blind and low-vision people
(§5.2, p. 7)
Research question
- Does spatial audio, compared to mono audio, in video meetings improve turn-taking (H1), social presence (H2) and group task performance (H3)?
Method and data
- Design
- Controlled within-subjects experiment. Each group held two 30-minute video meetings, one with spatial audio and one with mono audio, counterbalanced for audio condition and task order. Participants were not told which condition was on.
- Prototype
- An experimental web-based video meeting application, similar to standard ones. Each participant saw the other four or five people in a row across the top of the screen and themselves centred below, and simply talked, wearing wired headphones. With spatial audio on, the application placed each person's voice in a simulated virtual room to match their left-to-right position on screen, so voices seemed to come from where people appeared. Built on the browser's Web Audio API and the Resonance Audio SDK with high-bitrate Opus audio. Compared with the same application in mono, the standard video meeting experience.
Figure 1: Video meeting layout used for spatial audio study. Self-view shows underneath a line of all other participants. When spatial audio was enabled, the left to right visual position of participants matched the left to right placement on the audio stage. From the paper, Figure 1, p. 1. - Setting
- A self-managed pseudo field study, in which participants joined from their own devices and network connections.
- Participants / data
- The initial pool was 213 US information workers at a large global technology company. The final sample was 75 participants (25 women and 49 men in the demographic data) in 15 groups and 30 meetings. Group members were strangers.
- Duration
- Two 30-minute meetings per group, each followed by a 15-minute questionnaire.
- Analysis
- 7-point Likert items on turn-taking (from Sellen) and social presence (Networked Minds and Temple Presence Inventory) compared with Wilcoxon signed-rank tests, gender as a moderator. Group task scores compared with paired t-tests. Open responses categorised by valence.
Findings
Conversations felt more interactive. Employees meeting in small groups of strangers rated their conversation as more interactive with spatial audio than with mono audio. Ease of participation improved only at the weaker 10% significance level. #
“it was easier for my brain to figure out who was saying something”
More like sharing a room. With spatial audio rather than mono, participants felt more that they were sharing the same space as the group, and found it easier to understand the thoughts of others. #
“It was the closest I've felt to being in the same room as others in a virtual meeting.”
Women and men gained differently. In exploratory comparisons, women gained mainly in social presence, such as feeling able to reciprocate when others spoke, and men mainly in turn-taking, such as ease of participation and interactivity. #
“spatial audio might play a role in making video meetings more inclusive for women”
Task scores were higher, but not significantly. Groups scored slightly better on the survival tasks with spatial audio, but not significantly. On the harder of the two tasks, an exploratory analysis found that spatial audio improved several perceptions, including being able to attend to one speaker at a time. #
“It is therefore possible that spatial audio benefits particularly challenging tasks that require more dynamic conversations (i.e., the desert task), and has no effect or a potentially distracting effect in easy tasks (i.e., the moon task).”
Not everyone wanted it. Fewer than half the participants preferred spatial audio and almost as many preferred mono. Many were uncomfortable wearing headphones, which spatial audio works best with. #
“[it] was cool, but probably not something I would enable”
Questions this paper answers
- Why would it matter where a voice seems to come from?
- In person, hearing where sound comes from helps people tell who is speaking, direct attention, pick out one voice among several and judge when to speak. Mono audio strips these cues. In video meetings, restoring them with spatial audio made conversations feel more interactive and more like sharing a room. (See §4.4, p. 6; §4.4, p. 6.)
- Could spatial audio make meetings more inclusive?
- Possibly. In exploratory analyses, women gained mainly in feeling present and able to reciprocate, men in ease of participation. We suggest it might also help neurodiverse and blind or low-vision people, but these indications need more research. (See §4.4, p. 5.)
- Does better meeting audio lead to better results?
- Not demonstrably yet. Groups scored slightly higher with spatial audio, but not significantly, in a test limited by sample size. It helped perceptions most on the harder task, and earlier research links lively, interactive talk to more good ideas, so demanding, discussion-heavy tasks may benefit most. (See §4.3, p. 5.)
Quantitative results
| Measure | Result | Where |
|---|---|---|
| Sample | 75 participants, 15 meeting groups, 30 meetings (initial pool 213) | §3.2, p. 3 |
| "The conversation seemed highly interactive" (S5), spatial vs mono | z=-2.281, p=0.023 | §4.1, p. 4 |
| "I found it easy to participate in the conversation" (S1), 10% level | z =-1.782, p=0.075 | §4.1, p. 4 |
| "I felt as if I were sharing the same space as the group" (S9) and "It was easy to understand the thoughts of others" (S13) | z=-2.687, p=0.007, and z=-2.405, p=0.016 | §4.2, p. 5 |
| Women vs men: ability to reciprocate when others spoke (S19), change in mean | 0.36 vs. 0.06; z = -2.162, p = 0.031 | §4.2, p. 5 |
| Men vs women: ease of participation (S1), change in mean | 0.30 vs. 0.04; z=-2.033, p=0.042 | §4.1, p. 4 |
Contributions and implications
- Audio has been underemphasised in theories of video meeting fatigue. Easier turn-taking with spatial audio might help reduce it. (§5.1, p. 6)
- Audio should be treated as essential to systems meant to create social presence.
auditory stimuli should be regarded as a necessary rather than simply valuable component of systems intended to induce social presence
(§5.1, p. 7) - Spatial audio is likely a faster and cheaper investment than video, but wireless devices need to support it, ideally via an open or cheaply licensed standard.
- Future work should add objective turn-taking measures, study layouts and sound positioning for larger meetings, and find which scenarios spatial audio suits. It is not one-size-fits-all.
Limitations
- Participants were US knowledge workers at one technology company, all strangers to each other, doing tasks that only approximated real workplace conversation.
- Large attrition limited statistical power, especially for task performance, which is scored per group. (§6, p. 8)
- The experimental application did not fix participants' positions, so people did not see the same person in the same place, which likely caused audio and location dissonance.
›Concepts
- Spatial audio
- Meant to recreate how sound is heard in person. As implemented here:
Spatial audio consists of variable-volume audio streams spatially separated based on user-controlled location in a virtual room.
(§1, p. 2) - Social presence
- Biocca's definition, which frames why audio matters for presence:
"the degree to which a user feels access to the intelligence, intentions, and sensory impressions of another"
(§2, p. 2) - Cocktail party problem
- Cherry's term, the basis of selective listening:
the" cocktail party problem" was defined as being able to understand" what one person is saying when others are speaking at the same time"
(§2, p. 2) - Survival task
- A group exercise (here, desert or moon survival) in which members agree which items matter most for survival, scored against expert rankings.
Survival problems are commonly used in research on social presence and turn-taking because they induce features of normal conversation by creating natural turn exchanges
(§3.4, p. 4)
Figures
How to cite
Kate Nowak, Lev Tankelevitch, John Tang, and Sean Rintel. 2023. Hear We Are: Spatial Audio Benefits Perceptions of Turn-Taking and Social Presence in Video Meetings. CHIWORK 2023. https://doi.org/10.1145/3596671.3598578
This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.