Automated mapping of competitive and collaborative overlapping talk in video meetings

Research brief

By Eleni Margariti, Sean Rintel, Brendan Murphy, and Abigail Sellen.

Takeaway: Transcript timing alone can partly map turn-taking in video meetings, showing who keeps or cedes the floor when talked over. But cooperative overlaps (backchannels like 'uh huh') and competitive bids for the floor can look alike, because timing without content cannot show what an overlap is doing.

Why this matters

What was known

Turn-taking has long been quantified, but mostly from audio corpora or small controlled experiments, and often without considering privacy constraints. most prior work focuses on audio conversations, builds upon speech and audio datasets [19, 21] or reflects on controlled experiments in a limited number of meetings (§2.2, p. 2)

What this paper adds

What is new is the input and the safeguard. Ordinary, de-identified Microsoft Teams transcripts are processed automatically, so turn-taking can be studied across an organisation's real meetings without audio recordings and without anyone reading what was said. (§2.2, p. 2)

Who it matters to

Organisations wanting insight into meeting practices and inclusion, and researchers building meeting analytics for remote and hybrid meetings. Meeting analytics may assist organisations improve meeting practices by providing insights on turn-taking behaviour. (§1, p. 1)

Research question

• How can cooperative competitive overlaps be automatically identified based on eyes-off processing of meeting transcripts?

Method and data

Design

Development of an automated process. Overlap patterns were first identified visually and with conversation-analytic methods on some of the authors' own meetings, then parametrised from turn-length statistics and applied automatically to the full dataset.

Setting

Regular project/status meetings held on Microsoft Teams in a global technology company, using Teams' automatic speaker-attributed, time-stamped transcripts, processed by a custom application that de-identified participants.

Participants / data

A convenience sample of three project groups, n∼52 participants. 10 meeting series, 34 meetings, recorded July-August 2021.

Analysis

Descriptive statistics of turn, overlap and overtake durations. Turns parametrised as speech of 3s or more. Overlaps and gaps below 0.24s (an estimated return delay) excluded as indeterminate. Probabilities of continuing or ceding a turn under different overlap patterns.

Findings

• Meetings move fast. In some of the authors' own Teams meetings, used to develop the method, the median transcribed turn lasted under two and a half seconds (a fast-paced conversation). The transcription service also produced many very short utterances of one or two words. #

  (§4.1, p. 3)

• Timing cannot always reveal intent. Working from timing and speaker labels alone, some overlap patterns could be classed with reasonable confidence. For example, a long overlap at the start of a turn strongly indicates competition for the floor. Others stayed ambiguous: #

  “Pattern 6, a repeated overlap, may be a competitive attempt to take a turn or a cooperative backchannel.”

• Uninterrupted speakers usually carry on. Across all 34 meetings, a speaker whom nobody talked over was about twice as likely to keep talking as to hand over to someone else. #

  (§4.2, p. 6)

• One overlap changes the odds. When one other participant talked over a speaker once, the speaker became almost as likely to cede the turn to that person as to continue, with ceding slightly less likely. #

  “This illustrates that the impact of a single overlap is significant.”

• Persistent overlapping makes a takeover more likely. When one person talked over a speaker repeatedly within the same turn, that person taking over became more likely than the speaker carrying on. Such repeated overlaps may be bids for the floor, some of them rejected. #

  “This case includes a potential attempt to take turn, and whether an overtake is achieved or not.”

• Hints of individual turn-taking styles. In one meeting series, participants differed sharply. Those most likely to keep their turn when talked over once were not the same as those who kept it under repeated overlaps. Consistent differences like these point towards individual turn-taking profiles. #

  “In the case of multiple overlaps occurring (Figure 3 C), the same participants presented the opposite behaviour, with P3 always abandoning turn”

Questions this paper answers

Can software detect interruptions in video meetings without anyone reading what was said?

Partly. The method uses only the timing and speaker labels in automatic meeting transcripts, never their content, keeping it privacy-preserving and scalable. These alone revealed recurring overlap patterns and their effect on who held the floor. Timing alone cannot always tell a supportive overlap from a bid to take over. (See §4.1, p. 3; §4.2, p. 6; §4.2, p. 6.)

Is talking over someone always an interruption?

No. Conversation analysts distinguish cooperative overlaps, such as 'mm' or 'yeah', which show attention or agreement, from competitive ones that take the floor. Repeated overlaps can be either (encouragement or insistent bidding for a turn), so the two have to be modelled together. (See §4.1, p. 3.)

Do people have their own styles of handling being talked over?

It appears so, though the evidence is thin. In one meeting series, the participant most likely to keep talking after a single overlap always gave up the turn under repeated overlaps, and also talked over others most. Whether such turn-taking profiles hold across meetings needs a much larger dataset. (See §4.2, p. 5.)

Quantitative results

Contributions and implications

• The method lays a basis for assessing meeting inclusion from metrics such as overtakes with overlap and rejected attempts to take a turn. This work provides the basis for assessing meeting inclusion based on a combination of metrics (§5, p. 7)
• Possible profiles of inclusive and non-inclusive participation, for individuals and groups, but only with meeting context such as function and roles. this should not be judged prematurely without further insight into the meeting's context (function, role of attendees) (§5, p. 7)
• Profiling needs careful ethical and privacy safeguards. Labelling behaviours can harm individuals. linking behaviours such competitiveness or domination with race or gender should be avoided (§5, p. 7)
• With in-room speaker attribution, the analysis could be extended to hybrid meetings to measure the impact of being remote or local. it may be possible to measure the impact of being remote or local in a hybrid meeting (§5, p. 7)

Limitations

• Results are promising but initial, from a small dataset that needs validation with a larger sample. (§7, p. 7)
• Speech-to-text and transcript timing constrain accuracy. Latency makes very small overlaps and gaps indeterminate.
• Other telemetry, such as mute/unmute, reactions, raised hands and meeting chat, was not used and could broaden understanding of turn-taking. application telemetry such as mute/unmute, reactions, raise hands, and, in particular, the use of the meeting's text chat (§7, p. 7)

Figures

How to cite

Eleni Margariti, Sean Rintel, Brendan Murphy, and Abigail Sellen. 2022. Automated mapping of competitive and collaborative overlapping talk in video meetings. CHI EA 2022. https://doi.org/10.1145/3491101.3519612

This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.