Research brief
Takeaway: Video meetings can convey attention better with shared space. Seated around one virtual table as live 3D video, people scored higher than in a grid on shared space and attention. Yet most preferred the familiar grid, likely because the 3D design was unsettled and needed manual control.
Why this matters
- What was known
- Grid video meetings make it hard to direct and communicate attention, and virtual 3D meeting spaces that could help struggle to represent the people in them.
Avatars face resistance, and 2D video is limited to near-frontal views, constraining the spatial layout.
(Abstract, p. 1) - What this paper adds
- Earlier spatial meeting systems used flat video cutouts, avatars or special hardware. We turn each person into live 3D video from one standard webcam and seat everyone in a shared virtual room. We then test it in group meetings against flat video in the same room and a standard grid.
Views can be rendered in real-time from a wide range of viewpoints for many-to-many communication.
(§1, p. 1) - Who it matters to
- Designers and researchers of videoconferencing and virtual meeting spaces, for whom the authors see a payoff despite the mixed results:
live 3D video in virtual meeting spaces holds great potential for reducing fatigue and improving inclusion and effectiveness in remote meetings
(§1, p. 2)
Research questions
- RQ1: Can we capture 3D views of people using commodity webcams and render them in a 3D virtual environment?
- RQ2: What advantages does user-controlled view direction confer for focusing user attention, and conveying direction of attention to other users?
- RQ3: Are there advantages to a 3D video representation of users in the 3D virtual environment?
Method and data
- Design
- Within-subjects experiment with three conditions: (A) a traditional 2D videoconferencing grid, (B) 3D video of each person in a shared 3D room, and (C) 2D video of each person in the same room. In each condition groups counted from 1 to 20 in turn, then held a five minute decision-making discussion, then completed a survey. Sessions ended with a recorded demonstration of higher-quality 3D video.
- Prototype
- A videoconferencing system that runs on a standard PC with one ordinary webcam. For each webcam frame it finds the user's face, and a neural network predicts the depth of every pixel and separates the person from the background. The colour and depth are streamed to the others, whose computers turn them into a live 3D figure. Each person sees a first-person view of a simple virtual room with everyone seated around a hexagonal table, and turns their view left or right with the arrow keys. The others see their figure turn too. Audio is spatial, through stereo headphones, and parts of the body the camera cannot see are filled with a stylised 'projector' effect. It is built with WebRTC streaming and the Unity engine, and runs at almost 30 frames per second on a high-end graphics card. It was compared with a standard video grid and with flat 2D video placed in the same room.
Figure 1: Our prototype videoconferencing system predicts 3D geometry corresponding to each user's regular camera feed, and positions all users in a coherent 3D virtual space that is rendered from a first-person perspective. Note: We obtained explicit permission from colleagues who were not study participants to include their images in this paper, because showing relative mutual orientation of the 3D representations in the prototypes is crucial to understanding the research. From the paper, Figure 1, p. 2. - Setting
- One-hour experimental sessions at a global technology company, each participant at their own PC and webcam.
- Participants / data
- 48 participants (37 males and 11 females) in 8 groups of 6.
- Duration
- One-hour sessions.
- Analysis
- Survey items (5-point Likert scale) compared between conditions with Kruskal-Wallis tests and pairwise Wilcoxon rank-sum follow-ups, treated as independent samples. Illustrated with participants' open-text comments.
Findings
A shared room felt like one. Groups of six colleagues met in a shared virtual room, seated around a table. They rated it higher than a standard video grid on every sense-of-shared-space item, whether people appeared as flat or 3D video. 3D video added only to the sense that people took up physical space. #
“felt like a more natural interaction around a table”
Easier to tell who looks at whom. In the shared room, where each person turned their own view and the others saw them turn, participants knew better than in the grid when others were looking at them or at each other. Ratings did not differ significantly between flat and 3D video, though some comments favoured 3D. #
“it was hard to tell the direction people were facing in [2D video] compared with [3D video]”
Fewer awkward pauses. The grid produced more awkward pauses than either shared-room version, and was worse than the room with flat video for knowing when someone wanted to speak. Other conversation-flow ratings did not differ significantly. #
“if [they're] about to interrupt someone, who's where or if someone wants to talk or not”
But the grid was more comfortable. Participants rated the grid more comfortable and less distracting than both shared-room versions. In the room they could see only a few others at once and had to turn with the arrow keys, and the live 3D video flickered with visible artifacts. #
“in real life you just turn your head or even just eyes and you can cover all persons sitting in same room”
Most still chose the grid. Asked which interface they preferred overall, a majority (26 of 48) chose the standard grid, often as more appropriate for business. Flat video in the shared room came second, which some saw as a workable compromise. #
“best of both compromise with current capabilities”
Quality, more than 3D itself, seems to be the hurdle. Live 3D video was not preferred over flat video in the shared room. But participants preferred a higher-quality recorded 3D demonstration over all three live interfaces, suggesting that the quality of 3D video is what holds it back. #
“I wished the 2D was 3D but without the distracting shaders/effects”
Questions this paper answers
- Why might video meetings need a sense of shared space?
- In person, space is a resource for talk. Where people sit and face shows who is attending to whom and who may speak next. One shared virtual table restored some of this, helping people tell who was looking at whom, with fewer awkward pauses than a grid. (See §5, p. 5; §5, p. 5; §5, p. 5.)
- Do people prefer 3D video meeting rooms to the normal video grid?
- Not yet. In group meetings, most people preferred the familiar grid, finding it more comfortable, less distracting and more businesslike. Having to turn their view to see others, and artifacts in the live 3D video, counted against the shared room. (See §5, p. 5; §5, pp. 6–7.)
- Is 3D video of people better than flat video in a virtual meeting room?
- Only for the sense that people took up physical space, and flat video was rated more business-appropriate. On conversational factors the two were judged similarly, and more people preferred flat video. A higher-quality recorded 3D demonstration was preferred over everything, so 3D video quality seems to be the limit. (See §5, p. 5; §5, p. 5.)
- Can an ordinary webcam produce live 3D video of a person?
- Yes, roughly. In each webcam frame, a neural network predicts depth and separates the person from the background. This let us stream live 3D video for many-to-many meetings on a standard PC. But people noticed flicker and artifacts, especially when they saw a person from the side. (See §5, p. 5; §5, p. 5.)
Quantitative results
| Measure | Result | Where |
|---|---|---|
| Shared understanding of who was next to whom, effect of condition | H = 89.41, p < 0.001 | §5, pp. 4–5 |
| Knowing when others were looking at me, effect of condition | H = 37.6, p < 0.001 | §5, p. 5 |
| Awkward pauses, effect of condition | H = 14.14, p = 0.001 | §5, p. 5 |
| Felt present with the other people, effect of condition | p = 0.759 (no significant effect) | §5, p. 5 |
| Comfortable meeting experience, effect of condition (grid better) | H = 30.38, p < 0.001 | §5, p. 5 |
| Overall preference: grid / 2D video in 3D space / 3D video in 3D space | 26 people [54%] / 13 people [27%] / 9 people [18%] | §5, pp. 5–6 |
Contributions and implications
- The 3D spatial conditions scored no lower than the grid on knowing when others wanted to speak, which we take to mean they kept conversational cues typically absent with avatars (avatars were not tested).
indicating that the 3D videoconferencing conditions were able to preserve conversational cues that are typically absent with avatar-based systems
(§6, p. 7) - Controlling view direction creates a tension. A wider field of view would remove friction, but it would also remove the attention signal that turning provides.
widening the field of view for participants would remove the reason for participants to change their first-person perspective view to attend to different parts of the scene
(§6, p. 7) - The quality of 3D video is key for its acceptability and usefulness in shared 3D spaces.
further suggesting that the quality of 3D video is key for acceptability and utility in shared 3D spaces
(§6, p. 7)
Limitations
- Condition order was not counterbalanced, and survey responses could not be linked across conditions, so tests assumed independent samples and quantitative results need caution.
Due to technical constraints and time, condition order was not counterbalanced across participants.
(§4, p. 4) - The live 3D representations had known quality limitations (flicker and artifacts).
A key issue with the 3D representations in the live setting was their limited quality.
(§5, p. 5) - We did not directly compare with avatar representations.
We did not directly compare to avatar representations in this study
(§6, p. 7)
›Concepts
- Spatiality
- The paper's opening premise, and what grid interfaces take away:
Space is not just where we talk, it's also a part of talk.
(§1, p. 1) - Hypergaze and flattening
- A term from Bailenson's account of video-call fatigue, summarised here as one problem of the video grid:
Hypergaze and flattening [2] are the subjective configuration of all meetings as presentations, leaving users feeling stared at and where people look as not representing true focus of attention.
(§2, p. 2) - 3D video from a single webcam
- Each person becomes live 3D geometry that can be viewed from many angles, using one ordinary webcam. The key step, for each frame:
pass the ROI to a feedforward convolutional neural network (CNN) that predicts depth for each image pixel, as well as a segmentation of the ROI into foreground and background regions
(§3, p. 3) - User-controlled first-person view
- Each participant turns their view with the arrow keys, and their rendered orientation turns for everyone else too, signalling where they are looking. (§3, p. 4)
Figures
How to cite
Tom Cashman, Tim Hutton, Martin de La Gorce, Tibor Takács, Antonio Criminisi, Milica Ðorđević, Goran Dubajić, Ðorđe Marjanović, Milena Okošanović, Vukašin Ranković, Ivan Razumenić, Bojan Roško, Teo Šarkić, Marko Skakun, Miloš Stojanović, Nikola Veličković, Predrag Jovanović, Payod Panda, Lev Tankelevitch, and Sean Rintel. 2024. An Equal Seat at the Table: Exploring Videoconferencing with Shared Spatial Context combined with 3D Video Representations. CHI EA 2024. https://doi.org/10.1145/3613905.3650903
This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.