Belonging There: VROOM-ing into the Uncanny Valley of XR Telepresence

Authors: Brennan Jones, Yaying Zhang, Priscilla N. Y. Wong, Sean Rintel
Venue: 2021 Conference on Computer Supported Cooperative Work
Free from the publisher

AThe study

Exploratory study with ten pairs, 20 people, in an open office at a global technology organisation. In each pair, one local and one remote person played two games with a standard Beam telepresence robot and with VROOM.

  1. Ten pairsEach pair had one local and one remote person, mostly researchers and software engineers with limited headset experience.
  2. Two games, two systemsPairs played a search game and a word guessing game with each system, in counterbalanced order.
  3. Paired interviewEach pair then took part in a semi-structured interview of about 30 minutes.
  4. AnalysisInterviews were coded and cross-referenced with key moments in video of the sessions.

BThe system

  1. An avatar on the robotThe local person wears a HoloLens augmented reality headset and sees a life-size 3D avatar of the remote person standing over a telepresence robot.
  2. A 360° viewThe remote person wears a virtual reality headset showing a live 360° view from a camera on top of the robot.
  3. Drive and gestureThe remote person drives the robot with a thumbstick and moves the avatar's head and arms with their own head and hand controllers.
  4. The avatar mirrorsThe avatar copies head turns and arm movements, moves its mouth with speech, and plays walking, idle and blinking animations.

VROOM was a technology probe assembled from existing parts. The standard robot shows the remote person's face on a screen and is driven from a desktop app.

CWhat the study found

Gesture, gaze and voicehow local and remote users could show and see attention

  • In these sessions, remote users pointed across the room more easily with VROOM's avatar arms.The gestures were crude, though. At close quarters, the standard robot's ordinary video conveyed finer gestures.
  • Local users could tell where the remote person looked from the avatar's head turns.Head turns sometimes made up for gestures the avatar could not show.
  • Half the local users relied mostly on the remote person's voice.Reasons were the avatar's unnatural look, its lack of facial expression and the headset's narrow view. Some remote users recognised themselves in the avatar.

Moving and belongingdriving, mutual awareness and the sense of belonging in the room

  • Remote users found VROOM's 360° view more immersive but drove less confidently.The standard robot's odd camera views showed its edges and heading, which gave remote users a more predictable sense of the robot's body.
  • Neither local nor remote users knew what the partner could see or express.Some gestures were missed or never tried. Remote users felt dependent on local partners rather than equal, more so with VROOM than the standard robot.
  • The paper argues VROOM imitated presence more closely yet left less sense of belonging.The authors hypothesise an Uncanny Valley of Telepresence, where unmet expectations of being there leave the remote person feeling less meant to be there.

DKey takeaway

Telepresence can offer remote people the comforts of belonging, even in unnatural, asymmetric ways. Imitating presence closely may create an uncanny valley of telepresence, raising expectations the system cannot meet.

Made for this site with AI assistance from the verified research brief. It is not the paper's own figure.

Abstract

The world is entering a new normal of hybrid organisations, in which it will be common that some members are co-located and others are remote. Hybridity is rife with asymmetries that affect our sense of belonging in an organisational space. This paper reports a study of an XR Telepresence technology probe to explore how remote workers might present themselves and be perceived as an equal and unique embodied being in a workplace. VROOM (Virtual Robot Overlay for Online Meetings) augments a standard Mobile Robotic Telepresence experience by (1) adding a virtual avatar overlay of the remote person to the local space, viewable through a HoloLens worn by the local user, through which the remote user can gesture and express themselves, and (2) giving the remote user an immersive 360° view of the local space, captured by a 360° camera on the robot, which they can view through a VR headset. We ran a study to understand how pairs of participants (one local and one remote) collaborate using VROOM in a search and word-guessing game. Our findings illustrate that there is much potential for a system like VROOM to support dynamic collaborative activities in which embodiment, gesturing, mobility, spatial awareness, and non-verbal expressions are important. However, there are also challenges to be addressed, specifically around proprioception, the mixing of a physical robot body with a virtual human avatar, uncertainties of others’ views and capabilities, fidelity of expressions, and the appearance of the avatar. We conclude with further design suggestions and recommendations for future work.

Research brief

By Brennan Jones, Yaying Zhang, Priscilla N. Y. Wong, and Sean Rintel.

Takeaway: Combining mobile robotic telepresence with mixed reality should aim to help remote and local people feel the remote person belongs there, not just is there. Imitating presence raises expectations that go unmet (an 'Uncanny Valley of Telepresence'), so designs should offer the comforts of belonging, even in unnatural, asymmetric ways.

Why this matters

What was known
In hybrid collaboration, remote people are more likely to be marginalised. Technologies such as mobile robotic telepresence, VR, AR and 360° video each address part of what remote workers lack, but none addresses all of it. Remote workers fundamentally lack five of the building block affordances for physical autonomy and affiliation available to local workers: mobility, embodiment, expressiveness, awareness, and presence. (§1, p. 2)
What this paper adds
Instead of giving both ends the same video link, our probe, VROOM, gives each side an experience suited to it. We compare it with a standard telepresence robot, not to pick a winner but to see how people account for each approach to belonging. From this we propose an 'Uncanny Valley of Telepresence', after Mori's Uncanny Valley. we argue that there is an 'Uncanny Valley of Telepresence' in which there a sharp dip in the continuous increase of both local and remote users' senses of belonging as a telepresence experience becomes more like physical presence (§1, p. 2)
Who it matters to
Designers and researchers of telepresence and XR systems for hybrid organisations, where some members are co-located and others remote. We conclude that XR Telepresence should focus on providing the comforts of belonging, even if done in unnatural ways, rather than focusing on pure imitation. (§1, p. 3)

Research questions

Method and data

Design
Exploratory study. Pairs (one local, one remote) played a search game (Gather) and a word-guessing game (Heads Up) in two conditions, a standard Beam mobile robotic presence system and VROOM, in counterbalanced order with fixed roles. Followed by a semi-structured paired interview.
Prototype
VROOM (Virtual Robot Overlay for Online Meetings), a technology probe assembled from existing parts. The local person wears an AR headset (Microsoft HoloLens) and sees a life-size 3D avatar of the remote person standing over a Beam telepresence robot, whose screen is turned off. The avatar's face is made from a photo of the remote person. The remote person wears a VR headset showing a live 360° view from a camera on top of the robot, and sees their own avatar body from the shoulders down when they look down. They drive the robot with a thumbstick, and move the avatar's head and arms with their own head and two hand-held controllers. The avatar mirrors the remote person's head turns and arm movements, its mouth moves with their speech, and it plays walking, idle and blinking animations. Built in Unity, with printed markers on the robot so the headset can place the avatar. Compared with the standard Beam robot, which shows the remote person's face on a screen and is driven from a desktop app.
VROOM: a remote user in a VR headset drives a telepresence robot via 360° video, and a local user in AR glasses sees the remote user's life-size avatar overlaid on the robot.
Fig. 1. An overview of the VROOM system. From the paper, Fig. 1, p. 2.
Setting
Open-office space in the regional office of a global technology organisation.
Participants / data
Ten pairs, 20 participants in total, aged 25-54 (M = 35, SD = 7.3). Eight women and 12 men. Mostly researchers and software engineers, with varied and mostly limited experience of VR and head-mounted displays.
Duration
One session per pair: both game rounds plus an approximately 30-minute interview.
Analysis
Open, axial and selective coding of interview transcripts, video-based interaction analysis of the sessions, and cross-referencing of interview codes with key video events.

Findings

Questions this paper answers

Can a remote person point and gesture through a telepresence robot?
Better with a body than a screen. When ten pairs played games in an office, remote partners moving a life-size AR avatar overlaid on the robot could point across the room. Its head turns showed where they looked. But gestures were crude. Up close, the robot's video showed finer ones. (See §5.3, p. 17; §5.3, p. 18.)
Is a more realistic telepresence system always better?
Not necessarily. We hypothesise an 'Uncanny Valley of Telepresence'. Belonging rises as a system imitates physical presence more closely, until both sides expect more of the remote person than the system can deliver. Then belonging dips. Overcoming it means finding what makes belonging with others comfortable, not just imitating presence. (See §6, p. 22.)
Why is it hard to collaborate through a telepresence avatar?
Because neither side knew what the other could see or express. Local partners' headsets had a narrow view that cut off the avatar's gestures, and remote users did not know how their gestures appeared, so some stopped gesturing. Remote users also felt dependent on their local partners rather than equal. (See §5.1, p. 14; §5.4, p. 21.)
Why might an immersive view make a robot harder to drive?
Seeing is not moving. A 360° headset view made remote users feel immersed, but they still steered a robot whose edges they could no longer see. The standard robot's odd but predictable camera views, which showed its edges and where it was heading, gave a better sense of its body. (See §5.2, p. 16.)

Contributions and implications

Limitations

›

Concepts

Asymmetric XR telepresence
Rather than giving both sides the same view, VROOM fits each side's experience to its own endpoint: Instead of twinned endpoints, VROOM provides an experience contextualised to each endpoint (§1, p. 2)
Belonging
A remote person's ability to be part of a space: acting in it with autonomy (moving, pointing, being aware of others) and being affiliated with the people in it. (§1, p. 2)
Uncanny Valley of Telepresence
Our hypothesis, after Mori's Uncanny Valley. Past the turning point, unmet expectations make belonging drop: the higher the simulation level of a telepresence experience, or the more it tries to simulate or replicate actual presence, the greater the user's feeling of 'belonging to the space' , up to a turning point (§6, p. 22)
XR Telepresence (VROOM)
Remote presence that mixes augmented and virtual reality, with the local and remote person each given a different experience of the same encounter. VROOM is a bi-directional asymmetrical XR Telepresence system (§3, p. 6)
Proprioception
The sense of where one's body is. For remote users to feel they belong, a crucial step is that they transfer their sense of proprioception (the awareness of the position of movement) of their real body to the avatar/robot body (§5.2, p. 15)
Deictic gestures
The first of the gesture types we analyse: Deictic gestures are indicative or pointing movements (§5.3, p. 17)

Figures

VROOM: a remote user in a VR headset drives a telepresence robot via 360° video, and a local user in AR glasses sees the remote user's life-size avatar overlaid on the robot.
Fig. 1. An overview of the VROOM system. From the paper, Fig. 1, p. 2.
Illustrative curve of sense of belonging against simulation level: it rises from audio conferencing through video conferencing to the Beam robot, dips sharply where VROOM sits, then climbs steeply to physical in-person presence.
Fig. 15. The 'Uncanny Valley of Telepresence' hypothesis. While VROOM simulates in-person presence more than state-of-the-art tools like audio calling, video conferencing, and MRP, our findings suggest that it gave users less of a sense of 'belonging' to the space. Thus, VROOM is likely somewhere within the 'Uncanny Valley of Telepresence'. From the paper, Fig. 15, p. 22.

How to cite

Brennan Jones, Yaying Zhang, Priscilla N. Y. Wong, and Sean Rintel. 2021. Belonging There: VROOM-ing into the Uncanny Valley of XR Telepresence. 2021 Conference on Computer Supported Cooperative Work. https://doi.org/10.1145/3449133

This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.

BibTeX

@inproceedings{jones2021belonging,
author = {Jones, Brennan and Zhang, Yaying and Wong, Priscilla N. Y. and Rintel, Sean},
title = {Belonging There: VROOM-ing into the Uncanny Valley of XR Telepresence},
booktitle = {2021 Conference on Computer Supported Cooperative Work},
year = {2021},
month = {April},
publisher = {ACM},
url = {https://doi.org/10.1145/3449133},
doi = {10.1145/3449133},
}