Ad hoc adaptability in video-calling

Authors: Sean Rintel, Kenton O'Hara, Behnaz Rostami Yeganeh, Roman Rädle
Venue: 2015
Author's copy (PDF)

AThe paper

Position paper on bringing whatever devices are at hand into video calls. It draws on prior research and briefly reports a small study of ad hoc screen mirroring.

  1. Prior researchDecades of research on sharing resources in video calls, set against systems that still tie one user to one device.
  2. A mirroring studyA small study, reported in full elsewhere, of calls where anyone could mirror a phone to the shared display.
  3. Design argumentObservations from the study are read through accountability and intelligibility, meaning everyone can see and make sense of what the system is doing.

BThe prototype

  1. Show a codeAlongside the call on a shared display, a second window shows a QR code.
  2. Point a phoneAny participant, in the room or remote, points a phone at the code.
  3. Mirror the phoneThe phone's screen appears on the shared display, showing material on the phone or its live camera.phone materiallive camera
  4. One at a timeOnly one device mirrors at a time.

Before this, mirroring was usually limited to the person driving the host computer.

CWhat the paper argues

Observations from a small studyad hoc screen mirroring, reported briefly

  • In the mirroring study, people contributed to the call from where they sat in the room.People could also move around the room and beyond it while sharing from a phone.
  • In the mirroring study, individuals prepared material on phones while the group talked.Preparing in parallel let individual, subgroup and full group sharing interleave more fluidly.
  • In the mirroring study, participants sometimes negotiated who would share next.Only one phone could mirror at a time. Participants agreed turns on the spot or set an order in advance.
  • In the mirroring study, pointing a phone at the code showed the room someone would share.A remote participant's move to share could go unnoticed unless someone in the room was watching the screen.

The design argumentwhat calls across devices need

  • The paper argues most video calls make even simple device combinations difficult.Most video calling systems allow one username per device per call, so combining devices is cumbersome.
  • The paper argues device connections should be visible and intelligible to everyone in the call.Visible seams keep people from being confused or surprised by a connection. Design must also allow for remote participants seeing less.

DKey takeaway

Video calls can meet unplanned sharing needs by letting people bring in any device at hand. Visible seamfulness, showing every connection to everyone, remote participants included, may keep sharing understandable.

Made for this site with AI assistance from the verified research brief. It is not the paper's own figure.

Abstract

In this position paper we explore ad hoc adaptability across devices in video-calling. We note the current difficulty of even simple combinations, discuss design issues, briefly report on a study of ad hoc screen mirroring, and note future directions.

Research brief

By Sean Rintel, Kenton O'Hara, Behnaz Rostami Yeganeh, and Roman Rädle.

Takeaway: Cross-device interaction (bringing whatever devices are at hand into a call) can expand what people do in remote and hybrid meetings, because many sharing needs cannot be planned. But it needs 'seamful' connections that everyone, including remote participants, can see.

Why this matters

What was known
Despite decades of research on ad hoc sharing of resources in video-mediated communication, mainstream video-calling systems still tie one user to one device per call, so combining devices is cumbersome. the dominant free and commercial systems on the market are still largely funneled and siloed (§Introduction, pp. 1–2)
What this paper adds
We argue for seamfulness in multi-device video calls, against the 'it just works' ideal of invisible connection. Connections should be accountable and intelligible to everyone. A small study of screen mirroring illustrates the argument. accountability and intelligibility are the paramount drivers of designing for ad hoc adaptability (§Designing for ad hoc adaptability, p. 3)
Who it matters to
Designers and builders of cross-device video-calling systems, as web and device-connection standards open the way beyond host-centric architectures. The design of such complex systems will need to need to foreground accountability and intelligibility to balance simplicity with seamfulness. (§Conclusions, p. 4)

Research question

Method and data

Design
Position paper drawing on prior research and a small-scale study of ad hoc screen mirroring in video calls (reported in a separate paper).
Prototype
A screen-mirroring add-on for video calls on a shared display. Alongside the call, a secondary window shows a QR code. Any participant, in the room or remote, can point their phone at the code to mirror the phone's screen to the shared display, showing either material on the phone or its live camera. Only one device mirrors at a time. Previously, mirroring was usually restricted to the person driving the host computer.
Meeting room with a video call on a wall display showing a QR code. One person points a smartphone at the QR code to mirror the shared screen while another sits at the table.
Figure 4: Pointing a smartphone at the QR code provided a seamful way for participants to understand that one participant would soon be mirroring. From the paper, Figure 4, p. 3.
Setting
Video calls with a shared display, with some participants in the room and some remote.
Analysis
The study's analysis is not described here. Its observations are reported briefly and interpreted through accountability and intelligibility.

Findings

Questions this paper answers

Why is it so awkward to combine several devices in one video call?
Because most video-calling systems assume one username per device per call. A group in a room therefore clusters around one computer and shares one display at a time. Even one person's own devices cannot easily work together, so combining them often costs more effort than it is worth. (See §Designing for ad hoc adaptability, p. 3.)
What changes when anyone in a meeting can share to the main screen from their own phone?
When everyone on a call, in the room or remote, could mirror a phone to the shared display, sharing grew more fluid across individuals, subgroups and the whole group. People contributed from where they were, prepared material in parallel, and negotiated turns. This worked best for people in the room. (See §Designing for ad hoc adaptability, p. 3; §Designing for ad hoc adaptability, p. 3; §Designing for ad hoc adaptability, p. 3; §Designing for ad hoc adaptability, p. 3.)
Should connecting devices in a meeting be seamless and invisible?
Not entirely. We argue for visible 'seamfulness' (connecting should be easy, but visible, so nobody is surprised). When people connected a phone to a shared screen, the visible act told others in the room that someone was about to share. Remote participants' attempts were easier to miss. (See §Designing for ad hoc adaptability, p. 3.)

Contributions and implications

Limitations

›

Concepts

Ad hoc adaptability
What a video call should be able to take in as a task unfolds: Many tasks are achieved through a combination of prepared resources which are brought to bear as situationally appropriate, discovering the need for unexpected resources, and developing resources within the task itself. (§Introduction, pp. 1–2)
Accountability
In the ethnomethodological sense, social order achieved in the moment by making reasoning observable and reportable. For systems: System accountability should operate in the same way, manifested through intelligibility, which is an ongoing awareness of the state of the system. (§Designing for ad hoc adaptability, p. 3)
Seamfulness
Against the 'it just works' trend toward invisible connection, we argue for visible seams: there is value, too, in visible 'seamfulness', such that users are not confused or surprised by any given connection (§Designing for ad hoc adaptability, p. 3)
'One username per device per call' model
The root of the limits on combining devices in a call: These limitations are the result of the 'one username per device per call' model that pervades most videocalling architectures. (§Introduction, pp. 1–2)

Figures

Illustration of a woman in a busy cafe on a video call, holding a smartphone to her ear for audio while her laptop camera provides the video.
Figure 1: Personal combinations: Using a smartphone for audio and laptop for video in a noisy environment. From the paper, Fig. 1, p. 2.
Illustration of a living room: parents talk to grandparents on a laptop while two children play nearby, with extra devices and the TV linked into the call.
Figure 2: Domestic combinations: Using additional personal or shared devices to allow children to play in their own space while parents converse with grandparents from a laptop. From the paper, Fig. 2, p. 2.
Meeting room with a video call on a wall display showing a QR code. One person points a smartphone at the QR code to mirror the shared screen while another sits at the table.
Figure 4: Pointing a smartphone at the QR code provided a seamful way for participants to understand that one participant would soon be mirroring. From the paper, Figure 4, p. 3.

How to cite

Sean Rintel, Kenton O'Hara, Behnaz Rostami Yeganeh, and Roman Rädle. 2015. Ad hoc adaptability in video-calling. ITS'2015 - Workshop on Interacting with Multi-Device Ecologies in the Wild.

This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.

BibTeX

@misc{rintel2015ad,
author = {Rintel, Sean and O'Hara, Kenton and Rostami Yeganeh, Behnaz and Rädle, Roman},
title = {Ad hoc adaptability in video-calling},
year = {2015},
month = {November},
url = {https://seanrintel.com/publications/ad-hoc-adaptability-in-video-calling.html},
note = {ITS'2015 - Workshop on Interacting with Multi-Device Ecologies in the Wild},
}