What was Hybrid? A Systematic Review of Hybrid Collaboration and Meetings Research

Authors: Thomas Neumayr, Banu Saatci, Sean Rintel, Clemens Nylandsted Klokmose, Mirjam Augstein
Venue: 2021
Preprint (arXiv)

AThe study

A systematic literature review of research on hybrid collaboration and meetings, where co-located and remote people work together. It covers papers in the ACM Digital Library published up to November 2019.

  1. SearchThe ACM Digital Library was searched on 8 November 2019 with 72 keyword combinations.
  2. Screen 1,209 resultsTwo authors screened all 1,209 search results, and 108 looked potentially relevant.
  3. Read 108 in fullThe full texts of the 108 papers were checked for eligibility.
  4. Include 62 papersThe final set held 62 papers, the earliest from 1988.
  5. Code each paperEach paper was coded for terms, domains, study types and hybrid setting, then placed in a taxonomy.

BWhat the study found

A scattered fieldnames and volume

  • The 62 reviewed papers used 91 different terms for hybrid work.Of those 91 terms, 82 appeared only once. The word hybrid itself was rarely used before the pandemic.
  • Of 1,209 search results, 62 papers were about hybrid settings.That is 5.13%, so about 95% of results were false positives. Publications on hybrid settings declined after 2014.
  • Key research on Media Spaces fell outside the review's keyword search.Media Spaces were computing and video settings for unstructured collaboration across distance. That research assumed mixed local and remote groups without calling them hybrid.

How it was studiedmethods and focus

  • Of the 62 reviewed papers, 34 relied on experiments alone.Controlled experiments made up 21 of these and naturalistic experiments 13. Only 15 papers studied real-world work alone.
  • In 28 reviewed papers, the hybrid setting was present but left implicit.Another 26 made it explicit, one had both, and seven were not applicable, which accounts for all 62. Many papers included a hybrid setting without making it a focus.

Settings studied and missedsize, timing and informal contact

  • Reviewed studies mostly covered small groups across two locations.Most involved groups of around ten people or fewer. The review's authors note that real workplaces hold hybrid meetings with larger crowds and more locations.
  • Of 45 papers rated for timing, 32 focused on real-time interaction.Real-time interaction means people working together at the same moment. These timing ratings are interpretative and best read as rough indicators.
  • The review argues informal encounters are where hybrid design differs most from remote design.The gap between people present and people remote is especially pronounced there. An early always-on video wall linking two floors ran into privacy problems.

CKey takeaway

Hybrid collaboration has long been studied under many names, so its lessons are easily lost. Defining hybridity as asymmetry lets research build on what is already known.

Made for this site with AI assistance from the verified research brief. It is not the paper's own figure.

Abstract

Interest in hybrid collaboration and meetings (HCM), where several co-located participants engage in coordinated work with remote participants, is gaining unprecedented momentum after the rapid shift in working from home due to the COVID-19 pandemic. However, while the interest is new, researchers have been exploring HCM phenomena for decades, albeit dispersed across diverse research traditions, using different terms, definitions, and frameworks. In this article, we present a systematic literature review of the contexts and tools of HCM in the ACM Digital Library. We obtained approximately 1,200 results, which were narrowed down to 62 key articles. We report on the terms, citations, venues, authors, domains, study types, and data of these publications and present a taxonomic overview based on their reported hybrid settings’ actual characteristics. We discuss why the SLR resulted in a relatively small number of publications, and then as a corollary, discuss how some excluded high-profile publications flesh out the SLR findings to provide important additional concepts. The SLR itself covers the ACM until November 2019, so our discussion also includes relevant 2020 and 2021 publications. The end result is a baseline that researchers and designers can use in shaping the post-COVID-19 future of HCM systems.

Research brief

By Thomas Neumayr, Banu Saatci, Sean Rintel, Clemens Nylandsted Klokmose, and Mirjam Augstein.

Takeaway: A systematic review of CSCW research finds hybrid collaboration (co-located and remote people working together) scattered under dozens of names, its asymmetries often assumed rather than studied. Most research is experimental. Problems it had already identified, like remote people becoming an out-group, resurfaced when hybrid work surged during the pandemic.

Why this matters

What was known
Researchers have studied people in a room working with remote colleagues for decades, but under many different terms ('distributed', 'virtual', 'remote', 'mediated'), so there was no shared understanding of hybrid work. the emergent concepts on what we might want to call hybrid collaboration and meetings do not reflect a common understanding, conceptualization, and definition of phenomena that the world is now desperately racing to understand (§1, p. 2)
What this paper adds
A systematic literature review of hybrid collaboration and meetings in the ACM Digital Library up to November 2019, with a taxonomy of the hybrid settings studied. It adds a synthesis with excluded high-profile work (such as Media Spaces) and 2020 and 2021 publications. The end result is a baseline that researchers and designers can use in shaping the post-COVID-19 future of HCM systems. (Abstract, p. 1)
Who it matters to
Researchers, designers and developers of hybrid collaboration and meeting systems, and anyone searching the earlier literature on hybrid work. (§1.4, p. 5)

Research questions

Method and data

Design
Systematic literature review following Kitchenham and Charters, with a PRISMA flow, plus a narrative synthesis of excluded high-profile work and 2020 and 2021 publications.
Setting
The ACM Digital Library (ACM Full-Text Collection), queried on November 8th, 2019, with 72 keyword combinations (adjectives such as hybrid, partially distributed, virtual, video-mediated, crossed with nouns such as collaboration, meeting, team).
Participants / data
1,209 results screened. 108 potentially relevant. 62 publications in the final set.
Duration
Publications up to November 2019 (the earliest included paper is from 1988).
Analysis
Two main authors screened and coded papers for terms, citations, venues, authors, domains, study types, data collection and hybrid setting. Settings were classified in a taxonomy by participants' location, synchronicity and tool and device usage. Combining the SLR with additional literature, we proposed seven focal points for future work.

Findings

Questions this paper answers

Was there research on hybrid meetings before COVID-19?
Yes, going back to a 1988 paper, but not much of it and rarely called 'hybrid'. Of 1,209 ACM search results up to 2019, only 62 papers studied hybrid collaboration or meetings, and publications declined after 2014. (See §3.2, p. 19; §7, p. 49.)
Why is it hard to find research on hybrid work?
Researchers used many different terms (91 across 62 papers), most only once. Important work such as Media Space research also assumed mixed local and remote groups without naming them, so keyword searches miss a lot. (See §3.2, p. 19; §5.1, p. 41.)
What kinds of hybrid settings have researchers studied?
Mostly experiments with small groups, usually around ten people or fewer, across two locations and focused on real-time interaction. Only 15 of 62 papers were purely real-world studies. (See §3.6, p. 22; §7, p. 49.)
Is designing for hybrid work the same as designing for remote work?
Not entirely. For the mechanics of communication, designs that support hybrid groups usually work for fully remote ones too. But informal, spontaneous encounters differ, because the gap in awareness between people in the office and those working remotely is especially pronounced. (See §6.5, p. 47.)

Quantitative results

MeasureResultWhere
Search results and final corpus1,209 results; 62 relevant publications (5.13%; approximately 95% false positives)§2.2, p. 16
Different terms used to describe hybrid phenomena91 terms in 62 papers; 82 used only once§3.2, p. 19
Study types34 of 62 experiment-based only (21 controlled, 13 naturalistic); 15 purely real-world studies§3.6, p. 22
Hybrid setting explicit vs implicit26 explicit, 28 implicit, 1 both, 7 not applicable§3.6, p. 23
Focus on synchronous interaction32 of 45 publications categorised§4.1, p. 27
Citations of included papers (Google Scholar)mean 100.37, median 21.5§3.3, p. 20

Contributions and implications

Limitations

›

Concepts

Hybrid collaboration and meetings (HCM)
Our umbrella term, reviewing collaboration and meetings together because their boundaries are fuzzy. Both are settings where several co-located participants engage in coordinated work with remote participants (Abstract, p. 1)
Hybrid collaboration
It involves more than two people (who may split into subgroups) and several tools and devices at once. Above all, Hybrid collaboration switches back and forth between all four quadrants of the time-space matrix. (§1.1, pp. 3–4)
Hybrid meetings
Our working definition: we define hybrid meetings as video- or audio-based meetings among co-located and remote participants (§1.2, p. 4)
Time-space matrix
Johansen's classification of group activity as same or different time and same or different place. The taxonomy treats synchronicity and asynchronicity as separate continua.
Media Space
Stults's term for a computing and video setting for unstructured collaborative work among people separated by space and time. Deliberately not used as a search term, and discussed separately. Media Space settings were intended to naturally and seamlessly include hybrid cohorts, so there was no need to describe them as such (§5.1, p. 41)
Hybridity as asymmetry
Our organising lens. We prefer 'hybrid' because it best connotes an uneven split between co-located and remote cohorts, and the literature we review mostly treats that asymmetry as a source of problems, such as social barriers between the two groups: our goal is to unpack how hybridity matters when it confers an asymmetry on the coordination that occurs within the interrelated concepts of collaboration and meetings (§1.4, p. 5)

Figures

PRISMA flow diagram: 1,209 records identified through ACM Digital Library searching and screened, 108 full-text articles assessed for eligibility, and 62 studies included.
Fig. 1. PRISMA flow diagram according to [127]. From the paper, Fig. 1, p. 17.
Taxonomy table of reviewed papers: for each, group size, number of locations, synchronicity and asynchronicity bars, software tools, hardware devices, topics and comments. Most have two or three locations and strong synchronicity.
Fig. 4. The taxonomy of HCM (Part 1 of 2). From the paper, Fig. 4, p. 24.

How to cite

Thomas Neumayr, Banu Saatci, Sean Rintel, Clemens Nylandsted Klokmose, and Mirjam Augstein. 2021. What was Hybrid? A Systematic Review of Hybrid Collaboration and Meetings Research. arXiv. https://doi.org/10.48550/arXiv.2111.06172

This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.

BibTeX

@unpublished{neumayr2021what,
author = {Neumayr, Thomas and Saatci, Banu and Rintel, Sean and Klokmose, Clemens Nylandsted and Augstein, Mirjam},
title = {What was Hybrid? A Systematic Review of Hybrid Collaboration and Meetings Research},
year = {2021},
month = {November},
url = {https://doi.org/10.48550/arXiv.2111.06172},
note = {arXiv},
doi = {10.48550/arXiv.2111.06172},
}