Large Scale Analysis of Multitasking Behavior During Remote Meetings

Honorable Mention
Authors: Hancheng Cao, Chia-Jung Lee, Shamsi Iqbal, Mary Czerwinski, Priscilla Wong, Sean Rintel, Brent Hecht, Jaime Teevan, Longqi Yang
Venue: CHI 2021

AThe study

Mixed-methods study at a global technology company in 2020. It combined Microsoft Teams telemetry from 34,524 meeting attendances by US employees with a diary study of 715 employees worldwide.

  1. Meeting telemetryAnonymised records of email sent and files edited during video meetings, in four week-long snapshots from February to May 2020.
  2. RegressionThe odds of emailing during a meeting were compared by meeting length, size, type, hour and weekday.
  3. Diary studyOf 849 employees who signed up, 715 made at least one diary entry between mid-April and mid-August 2020.
  4. Coding diariesResearchers openly coded 100 randomly selected answers from 819 about multitasking, for its reasons and consequences.

BWhat the study found

In the telemetrymeeting features linked to emailing during meetings

  • In the telemetry, about 30% of meeting attendances involved sending email during the meeting.The rate was 31.1% in February and 28.9% in May 2020. Remote work effects could not be separated from COVID-19 effects.
  • In the telemetry, longer meetings were linked to higher odds of emailing.Meetings over 80 minutes had 6.21 times the odds of emailing of meetings up to 20 minutes long.
  • In the telemetry, larger meetings were linked to higher odds of emailing.Meetings with more than 10 attendees had 2.16 times the odds of emailing of meetings with one or two attendees.
  • In the telemetry, morning and recurring meetings were linked to more emailing.Morning meetings had 1.86 times the odds of after-hours meetings. Recurring meetings had 1.59 times the odds of ad hoc ones.

In the diariesreasons and consequences employees reported

  • Of coded diary answers, 39% said people multitasked to catch up on work.Self-reported. Other reasons included meetings that seemed irrelevant to the person, pop-up notifications and distractions at home.
  • Of diary answers, 32% linked multitasking to turning off video or muting.Self-reported. Turning off video or muting made it easier to go unnoticed.
  • Of coded diary answers, 15% said multitasking boosted productivity.Self-reported, especially when the side work served the meeting, such as taking notes or looking up information under discussion.
  • Of coded diary answers, 36% named loss of attention as a downside.Self-reported. It was the downside mentioned most often, alongside mental fatigue and seeming rude to colleagues.

CKey takeaway

How remote meetings are scheduled and structured can predict when people multitask. Treating attention as a spectrum may let meetings make room for positive multitasking while protecting engagement.

Made for this site with AI assistance from the verified research brief. It is not the paper's own figure.

Abstract

Virtual meetings are critical for remote work because of the need for synchronous collaboration in the absence of in-person interactions. In-meeting multitasking is closely linked to people’s productivity and wellbeing. However, we currently have limited understanding of multitasking in remote meetings and its potential impact. In this paper, we present what we believe is the most comprehensive study of remote meeting multitasking behavior through an analysis of a large-scale telemetry dataset collected from February to May 2020 of U.S. Microsoft employees and a 715-person diary study. Our results demonstrate that intrinsic meeting characteristics such as size, length, time, and type, significantly correlate with the extent to which people multitask, and multitasking can lead to both positive and negative outcomes. Our findings suggest important best-practice guidelines for remote meetings (e.g., avoid important meetings in the morning) and design implications for productivity tools (e.g., support positive remote multitasking).

Research brief

By Hancheng Cao, Chia-Jung Lee, Shamsi Iqbal, Mary Czerwinski, Priscilla Wong, Sean Rintel, Brent Hecht, Jaime Teevan, and Longqi Yang.

Takeaway: How remote meetings are scheduled and structured predicts when people multitask. Long, large, recurring and morning meetings draw more side work, often because topics don't concern attendees or they need to catch up. Treating attention as a spectrum can let multitasking aid productivity rather than only cost engagement.

Why this matters

What was known
Earlier work on multitasking in meetings and video chats focused mostly on in-person settings and on people's perceptions. these works are mainly based on small-scale qualitative studies, and to the best of our knowledge, no research to date has reported systematic and comprehensive evidence from large populations (§1, p. 2)
What this paper adds
A large-scale, mixed-methods study combining anonymised telemetry with a diary study, linking meeting characteristics to multitasking and describing its motivations and consequences. We present the first large-scale, empirical study of multitasking behavior during remote meetings, accompanied by rich qualitative evidence. (§1, p. 2)
Who it matters to
Meeting organisers, who can use best-practice guidelines, and designers of productivity and remote collaboration tools.

Research questions

Method and data

Design
Mixed methods: conditional logistic regression on anonymised telemetry (stratified by worker, standard errors grouped by meeting), plus a diary study.
Setting
A large technology company: Teams meetings, Outlook email and OneDrive/Sharepoint file edits of US employees, in four week-long snapshots from February (pre-COVID) to May 2020, and diary study of employees globally during COVID-19.
Participants / data
34,524 (meeting, user) records in the May 18-22 regression snapshot. 849 diary participants onboarded, of whom 715 completed at least one entry. 819 multitasking responses from 413 participants, of which 100 were randomly selected for coding.
Duration
Telemetry February to May 2020. Diaries mid-April to mid-August 2020, 24 requested entries over two months.
Analysis
Regression of email and file multitasking on meeting duration, size, type, hour of day and day of week. Open coding of diary responses, with five researchers independently coding the first 20% and one researcher coding the rest.

Findings

Questions this paper answers

How common is multitasking in video meetings?
Very common. In telemetry from a large technology company in early 2020, about 30% of meeting attendances involved sending email during the meeting. Employees also felt they multitasked more once meetings moved online. (See §4, p. 5.)
Why do people multitask more in big or long meetings?
People say they cannot concentrate through long meetings, and large meetings often cover topics that do not concern them, whereas small meetings demand their full attention to follow the conversation. In company telemetry, the odds of multitasking rose steadily with meeting size and length. (See §5.1, pp. 6–7; §5.2, p. 8.)
Is multitasking during meetings always bad?
No. Multitasking can help when it serves the meeting itself, such as taking notes or looking things up, and some people said it made them more productive. The main cost is attention. Losing track of the meeting was the downside mentioned most, along with fatigue and seeming rude. (See §7.1, p. 9; §7.2, pp. 9–10.)
Does turning your camera off make multitasking more likely?
It seems so. People often linked multitasking to turning off video or muting, since remote meetings make it easy to stay in the background unnoticed. Some kept their camera on precisely to resist the temptation, because multitasking on camera feels rude. (See §5.2, p. 8; §7.2, pp. 9–10.)

Quantitative results

MeasureResultWhere
Telemetry records in the regression34,524 (meeting, user) records§3.1, p. 4
Meetings with email multitasking, February to May 202031.1%, 30.9%, 29.2%, and 28.9%§4, p. 5
Odds of email multitasking by meeting size (3, 4-5, 6-10, >10 attendees vs one or two)1.12 (P = 0.021), 1.39 (P < 0.001), 1.70 (P < 0.001) and 2.16 (P < 0.001)§5.1, p. 6
Odds of email multitasking by duration (20-40, 40-80, >80 minutes vs 0-20 minutes)1.96, 3.22 and 6.21 (P < 0.001)§5.1, p. 6
Odds of email multitasking, morning and afternoon vs after hours1.86 (P < 0.001); 1.54 (P < 0.001)§5.1, pp. 6–7
Odds of email multitasking, recurring and scheduled vs ad hoc meetings1.59 (P < 0.001) and 1.31 (P = 0.012)§5.1, pp. 6–7
Diary responses linking multitasking to turning off video or muting32% of responses§4, p. 5

Contributions and implications

Limitations

›

Concepts

Multitasking
In meetings it can be related to the meeting (such as note taking) or unrelated (such as email). The general definition: Information work is often governed by multiple tasks and activities that an individual must remember to perform, often in parallel or in rapid succession, a practice that is called multitasking (§1, p. 1)
Email and file multitasking
Our telemetry proxies: sending, replying to or forwarding at least one email, or editing a productivity file, during a meeting. (§1, p. 2)
Positive multitasking
Multitasking that fits the meeting's purpose or helps people work, such as note taking or finding relevant information. The design catch: Some positive multitasking fits the meeting purpose, but technically requires moving outside the meeting window which may increase the risk of being distracted from the meeting. (§9, p. 11)

Figures

Five charts of the share of user-meeting pairs with email multitasking: rising with meeting duration and size, higher for scheduled, recurring and broadcast than ad hoc meetings, peaking in the morning, and lowest on Friday.
Figure 4: Proportion of user-meeting pairs with email multitasking versus meeting characteristics measured by telemetry data. People multitask more in longer meetings, larger meetings, recurring/scheduled meetings, morning meetings and Tuesdays. From the paper, Fig. 4, p. 8.
Three line charts of activity by hour of day for February to May 2020: email and file actions keep the same daily shape, while remote meeting volume more than doubles after February, with morning and afternoon peaks.
Figure 2: Transition of worker work rhythms (i.e., distribution of work related actions over time within a day) from Feb. 2020 (pre-COVID-19) to May 2020 on email, file editing actions and remote meetings. While email and file usage remains stable, there is a clear increase in the volume (over twice as much) of remote meetings after the breakout of the pandemic. Results are normalized using maximum volume in Feburary. From the paper, Fig. 2, p. 6.

How to cite

Hancheng Cao, Chia-Jung Lee, Shamsi Iqbal, Mary Czerwinski, Priscilla Wong, Sean Rintel, Brent Hecht, Jaime Teevan, and Longqi Yang. 2021. Large Scale Analysis of Multitasking Behavior During Remote Meetings. CHI 2021. https://doi.org/10.1145/3411764.3445243

This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.

BibTeX

@inproceedings{cao2021large,
author = {Cao, Hancheng and Lee, Chia-Jung and Iqbal, Shamsi and Czerwinski, Mary and Wong, Priscilla and Rintel, Sean and Hecht, Brent and Teevan, Jaime and Yang, Longqi},
title = {Large Scale Analysis of Multitasking Behavior During Remote Meetings},
organization = {ACM},
booktitle = {CHI 2021},
year = {2021},
month = {May},
url = {https://doi.org/10.1145/3411764.3445243},
doi = {10.1145/3411764.3445243},
}