Research brief
Takeaway: Telepresence robots let remote people roam a space but show little body language. A life-size avatar overlaid on the robot, seen through AR glasses, could show where the remote person looks and points. A VR headset gives them a live all-round view, with their avatar's body seen in first person.
Why this matters
- What was known
- Telepresence robots let remote users move around a space and give them a physical presence there, but they show the remote person only on a small 2D screen. AR avatars convey gestures but cannot roam.
However, they lack a compelling representation of the remote user in the local space.
(Abstract, p. 1) - What this paper adds
- Earlier mixed-reality work attached a miniature avatar to a 360° camera. We attach a life-size avatar to a telepresence robot instead, so the remote person can move through the space and keeps a physical presence that people without a headset can also see. (§Introduction, p. 2)
- Who it matters to
- Designers and researchers of telepresence and mixed-reality collaboration. We note that remote collaboration technologies can include people who would otherwise be left out, such as people with disabilities who cannot leave home or who cannot afford to travel.
Research question
- How to combine robotic telepresence with mixed-reality avatars to provide the best of both worlds to each endpoint of a video-call experience?
Method and data
- Design
- System design and implementation, illustrated with a usage scenario. No user study is reported. A comparison with standard robotic telepresence is left for a future paper.
- Prototype
- VROOM, a two-way system. In the local space, people wearing a Microsoft HoloLens (version 1) augmented-reality headset see a life-size avatar of the remote person overlaid on a BeamPro telepresence robot. Printed markers on the robot keep the avatar in place. The avatar's face is made from a photo of the remote person. Its head turns as theirs does, its mouth moves when they speak, its arms follow their hand controllers, and it walks when the robot drives. In the remote space, the remote person wears an HP Windows Mixed Reality virtual-reality headset showing live 360° video from a RICOH Theta V camera on top of the robot. They see their own avatar body when they look down or move their hands, and drive the robot with the controller thumbsticks. Both ends run Unity applications.
Figure 1: An overview of the VROOM system. From the paper, Figure 1, p. 2. - Setting
- A local space where the activity takes place, and a remote space from which one person joins through the robot.
- Analysis
- Not applicable (system description).
Findings
Gaze and pointing carry over. Because the avatar's head and hands follow the remote user's, people on site can see where the remote person is looking and what they are pointing at. Our usage scenario imagines a design director in Seattle moving around her team's studio in Shanghai. #
“Her colleagues, each wearing a HoloLens, can see her avatar present in any room, moving around the building, and if they call to her they can see her look back over her shoulder.”
One avatar, two audiences. The avatar is designed for both ends. On site, it should make the absent colleague feel there. At home, seeing one's own avatar body in the live 360° view should build a sense of being on site. #
“This full-body avatar is meant to heighten the local user[’s] sense that the remote user is present in the space with them.”
Simple tracking was enough for now. Markers on the front and back of the robot were enough to keep the avatar facing the same way as the robot in this first version, though smoother tracking may need more markers. #
(§Local Space, p. 5)The robot's screen could go. A future version could use any driveable robot with a 360° camera on a pole at head height. We kept the robot's screen only so that VROOM could later be compared with standard robotic telepresence. #
“the screen would be unnecessary in a space where all local users wore a headset”
Aimed at a known gap. Prior work found that 360° video gives remote people a wider, more immersive view but no easy way to gesture or show where they are looking. VROOM's avatar is our attempt to fill that gap, not yet evaluated. #
“simply adding a larger field of view does not provide an easy way of gesturing to other users or indicating which direction they are looking in the remote space”
Questions this paper answers
- How can a telepresence robot show the remote person's body language?
- One untested approach is to overlay an avatar of the remote person on the robot. In our system, VROOM, people on site wearing an augmented reality headset see a life-size avatar on the robot. Its head and hands follow the remote user's, showing where they look and point. (See §Usage Scenario, p. 5; §Local Space, p. 4.)
- Is a wider camera view enough for remote people to take part?
- Not on its own. Prior work found that 360° video gives a more immersive view but no easy way to gesture or show where one is looking. Our system aims to add both through an avatar that people on site can see. (See §Background, p. 3.)
- Does an avatar on a telepresence robot improve remote collaboration?
- We do not yet know. This paper describes the system and a usage scenario. How it affects social and spatial presence compared with standard robotic telepresence is future work. (See §Local Space, p. 5; §Background, p. 3.)
Contributions and implications
- A future version could drop the robot's screen and use any driveable robot with a 360° camera on a pole reaching head height.
- VROOM could be extended with a shared mixed-reality workspace where remote and local users place and handle virtual objects, for uses such as home planning or remote training.
we are interested in providing the local and remote users with a shared mixed-reality workspace
(§Future Work, p. 6)
Limitations
- No evaluation is reported. The comparison with standard robotic telepresence is left to a future paper.
to be reported in a future paper
(§Local Space, p. 5) - Avatar tracking used only front and rear markers, and smoother tracking may need more.
- Only one local and one remote user were illustrated. Multiple users, remote users seeing each other's avatars, and lower-cost robots remain to be explored.
›Concepts
- Asymmetries of video communication
- Differences between what remote and local people can see, control and do. A remote caller usually cannot steer their own view of the room, for example.
A remote user[’s] lack of ability to control their view can lead to frustration and provide an unequal experience
(§Background, pp. 2–3) - Telepresence robot
- A remotely driven robot with a screen, speakers, microphone and camera that lets a remote user move around and interact in another space.
they give remote users the ability to freely explore an environment that they are not physically in
(§Background, p. 3) - Physical embodiment
- What the robot's physical presence gives the remote user:
thus giving them a place in that environment, making that space belong to them
(§Background, p. 3) - Avatar overlay
- A life-size avatar of the remote user, built from a photo of their face and animated by their head and hand movements, shown on top of the robot to local users wearing AR headsets.
the mouth flaps in time with speech, and blink and idle animations are included to give the avatar some natural movement
(§Local Space, p. 4)
Figures
How to cite
Brennan Jones, Yaying Zhang, Priscilla Wong, and Sean Rintel. 2020. VROOM: Virtual Robot Overlay for Online Meetings. CHI 2020. https://doi.org/10.1145/3334480.3382820
This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.