The CoExplorer Technology Probe: A Generative AI-Powered Adaptive Interface to Support Intentionality in Planning and Running Video Meetings

Authors: Gun Woo (Warren) Park, Payod Panda, Lev Tankelevitch, Sean Rintel
Venue: DIS 2024
Free from the publisher

AThe study

Technology probe study with 26 employees of a global technology company. Each tried the CoExplorer prototype, driven by GPT-3.5 and GPT-4, in a simulated meeting, in a one-to-one online session of about one hour.

  1. Simulated meetingA fictional product team decides features for new headphones, with simulated attendees holding contrasting views.
  2. Guided walkthroughEach participant tried the prototype on their own laptop and thought aloud.
  3. InterviewParticipants discussed each design concept in a semi-structured interview.
  4. AnalysisTranscripts were coded for attitudes to each concept, then for the reasons behind them.

BThe tool

  1. Draft a planFrom a pasted calendar invitation, the AI shows each attendee a suggested goal, its reasoning and a list of phases.meeting goalreasoningphases
  2. Gather viewsThe AI generates a Meeting Focus Tool in which attendees state their preferences before the meeting.
  3. Refine the planThe AI narrows the goal and phases to where attendees' views diverge.
  4. Follow the talkDuring the meeting, the AI transcribes speech and detects when talk moves to a new phase.
  5. Check, then switchThe system asks whether anyone objects. If no one does, it brings up the video, files and apps for the new phase.

In the study, the plan and focus tool were generated in advance and the researcher triggered phase changes. There was no comparison group.

CWhat the study found

Before the meetinggoal, phases and focus tool

  • Of 26 participants, 24 found gathering attendees' preferences beforehand helpful.Participants said it would let discussion focus on where attendees disagreed. Of 26, 17 wanted the focus tool well in advance of the meeting.
  • Of 26 participants, 14 wanted the organiser to have the final say.The prototype refined the plan automatically. Some participants worried this could overwrite the organiser's agenda or let dominant speakers steer the meeting.
  • Of 26 participants, 7 valued how fast a line of text became a meeting structure.Ten were excited that the system gathered relevant files and apps for each phase. Six worried too much structure would make meetings rigid and linear.

During the meetingfiles and apps for each phase

  • Of 26 participants, 21 said the system should ask before showing files or apps.For phase changes, 16 of 26 said the same. The prototype acted unless someone objected.
  • Some participants worried that frequent prompts would distract during a meeting.The paper notes that asking before each change and interrupting less pull in opposite directions.
  • Of 26 participants, 20 expected a way to bring back windows the system closed.The system rearranges the files and apps on screen for each new phase.

Trust in the AIacross the walkthrough

  • Nearly every participant was sceptical of the AI's generated content, despite no visible errors.Even so, 17 of 26 trusted the content it gathered, often on conditions such as more experience with the system or previews of its choices.

DKey takeaway

Generative AI can support meeting intentionality by gathering attendees' views beforehand, so discussion focuses on where they disagree. People may trust it more when it asks before sharing files, leaves organisers the final say and rarely interrupts.

Made for this site with AI assistance from the verified research brief. It is not the paper's own figure.

Abstract

Effective meetings are effortful, but traditional videoconferencing systems offer little support for reducing this effort across the meeting lifecycle. Generative AI (GenAI) has the potential to radically redefine meetings by augmenting intentional meeting behaviors. CoExplorer, our novel adaptive meeting prototype, preemptively generates likely phases that meetings would undergo, tools that allow capturing attendees’ thoughts before the meeting, and for each phase, window layouts, and appropriate applications and files. Using CoExplorer as a technology probe in a guided walkthrough, we studied its potential in a sample of participants from a global technology company. Our findings suggest that GenAI has the potential to help meetings stay on track and reduce workload, although concerns were raised about users’ agency, trust, and possible disruption to traditional meeting norms. We discuss these concerns and their design implications for the development of GenAI meeting technology. KEYWORDS: video meetings, effectiveness, effort, design, adaptive user interface, windowing system, speech recognition; intent recognition, technology probe

Research brief

By Gun Woo (Warren) Park, Payod Panda, Lev Tankelevitch, and Sean Rintel.

Takeaway: Generative AI can ease meeting intentionality (knowing why you meet). From the invitation, AI can generate a goal, meeting phases and a poll of where attendees agree, focusing discussion on differences. Then an adaptive interface can open apps and files as talk changes phase.

Why this matters

What was known
Videoconferencing systems mostly connect endpoints. Research and commercial AI tools target specific pain points, missing the bigger picture of meeting intentionality across the meeting lifecycle. (§2.2, p. 3)
What this paper adds
A working probe that applies generative AI across the whole meeting lifecycle (from the invitation to in-meeting screen layouts) rather than to a single pain point. It comes with a user study, design implications and the prompts used, so others can rebuild it. (§1, p. 2)
Who it matters to
Designers and researchers building generative AI meeting technology and adaptive windowing systems. Our results and design implications suggest next steps for creating GenAI-mediated meeting systems, and the Appendix helps other researchers recreate and build upon our system. (§7, p. 13)

Research questions

Method and data

Design
Technology probe study: a guided walkthrough of the CoExplorer prototype in a simulated meeting, with a semi-structured interview and think-aloud.
Prototype
CoExplorer, a video meeting app driven by generative AI (GPT-3.5 and GPT-4, with Microsoft Azure speech-to-text). From a pasted calendar invitation, it shows each attendee a suggested meeting goal, the reasoning behind it and a list of phases (such as introduction, brainstorming and discussion). It also generates a Meeting Focus Tool (here a feature-ranking calculator) in which attendees state their preferences before the meeting. It then narrows the goal and phases to where views diverge. During the meeting it transcribes speech, detects when the talk moves to a new phase, and asks whether anyone wants to stop the change. If no one objects, it rearranges the screen into a tiled layout of video, files and apps for that phase. For the study, the phases and focus tool were pre-generated and the researcher triggered phase changes. There was no comparison condition.
CoExplorer screens: a generated goal, list of phases and reasoning beside a feature-selection calculator, and attendees' selections lead to a refined goal and phases focused on features where they disagreed.
Figure 4: (A) Sequence of phases and tool for determining meeting focus. (B) Participants in the meeting employ the focus tool to select preferred features. (C) CoExplorer uses the aggregated preferences to adjust the meeting's objective and flow. From the paper, Fig. 4, p. 6.
Setting
One-hour one-to-one sessions over Microsoft Teams, on participants' own laptops. A fictional scenario in which a cross-functional product team decides features for a new version of headphones, using fictitious documents. The other attendees were simulated, with contrasting opinions assigned to them.
Participants / data
26 employees of a large global technology company (18 males and 8 females) in roles including project manager, software engineer, hardware engineer, designer and researcher.
Duration
About one hour per session.
Analysis
Transcripts analysed with closed coding of attitudes to specific concepts, followed by iterative open coding of the reasons behind them.

Findings

Questions this paper answers

Would people want AI to plan their meetings?
Partly. AI can support meeting intentionality before a meeting. When employees tried a prototype, gathering attendees' views beforehand to focus discussion on goals was widely welcomed. Some valued quick structure from a short invitation. But many wanted the organiser to approve the AI's changes, and some feared rigid structure. (See §5.2, p. 9; §5.2.1, p. 9; §5.4, p. 10.)
Should AI act on its own during a meeting?
Mostly no. People liked the idea of a system finding relevant files and apps. But most wanted to be asked before anything appeared on the shared screen, while also not wanting frequent interruptions. Such systems have to resolve this tension. (See §5.4, p. 10; §5.4.1, p. 11.)
Do people trust AI meeting assistants?
Not readily. People who tried an AI meeting prototype mistrusted its generated content even when it had made no mistakes. This suggests an aversion that predates any error. Where they did trust it, it was often on condition of longer experience or previews of its choices. (See §5.4.1, p. 11; §6.1, p. 11.)

Quantitative results

MeasureResultWhere
Participants26 employees; 18 males and 8 females§4.1, p. 8
Found capturing attendees' preferences in advance helpful24 of 26§5.2, p. 9
Trusted the curated contents (often conditionally)17 of 26§5.4, p. 10
Expected a way to recall windows the system closed20 of 26§5.4, p. 10
Said the system should consult humans before showing files or apps; before phase changes21 of 26; 16 of 26§5.4.1, p. 10
Wanted the organiser to have the final say on attendees' input14 of 26§5.2.1, p. 9
Wanted the tool well in advance of the meeting17 of 26§5.2, p. 9
Appreciated the easy structuring capability7 of 26§5.2, p. 9
Excited about the system curating shared applications and files10 of 26§5.4, p. 10
Voiced concerns about excessive structuring of the meeting6 of 26§5.4.2, p. 10

Contributions and implications

Limitations

›

Concepts

Meeting intentionality
What the paper means by intentionality: everyone in a meeting knowing why they are there, what it is for, and what they need to discuss (§2.2, p. 3)
Meeting phases
Locally organised, activity-oriented stretches of a meeting that may match, split or span agenda items and need different resources. Phases are activity-oriented, often involving the use of multiple resources, and different phases may require different resources. (§2.1, p. 2)
Human-on-the-loop (HOTL)
The system acts on its own and humans can only abort its decisions, unlike human-in-the-loop, where actions need prior approval. HOTL characterizes humanmachine interactions in which the automated system mainly allows humans to abort the machine's decisions, once initiated by the system (§3.1, p. 5)
Meeting Focus Tool
A generated pre-meeting tool in which attendees state preferences, used to refine goals and phases towards points of disagreement. a Meeting Focus Tool that lets meeting participants assess the implications of including specific features in a product from their role perspective and voice their preferences on key features (§3.2.1, p. 7)
Technology probe
A working prototype used to elicit users' reactions to design concepts rather than to evaluate performance. our technology probe walk-through allowed users to engage with a working example of a system that was not yet possible to build to a fidelity necessary for plausible evaluation (§6.5, p. 13)

Figures

Diagram. Before the meeting: AI suggests phases from the invitation, attendees answer a meeting focus tool, and AI refines the phases. During the meeting: as talk moves to a new phase, the right apps and files open.
Figure 1: CoExplorer uses Generative AI to reduce the effort of meeting intentionality. From a natural language calendar invitation, GenAI is used to generate a meeting goal, a suggested list of meeting phases and their associated activities, and a meeting focus tool to help find specific needs for discussion. From attendee responses to that tool, GenAI refines the list of meeting phases. When the meeting is underway, GenAI is used to monitor the talk and either suggest or follow users' transitions between phases to fit the trajectory of the meeting. For each phase, GenAI generates apps and optimizes their position and size to fit the meeting phase. From the paper, Fig. 1, p. 1.
Diagram of the meeting timeline: a pre-meeting focus tool feeds suggested goal and phases (intro, brainstorming, discussion, conclusion), each with its own adaptive layout, and post-meeting is out of scope.
Figure 2: This framework shows the pre-, during, and post- meeting stages and identifies pain points (initiating focused discussion; resource management for each phase), and suggests points of intervention (meeting focus tool; adaptive phase definitions; layouts tailored to each phase). From the paper, Fig. 2, p. 5.
CoExplorer screens: a generated goal, list of phases and reasoning beside a feature-selection calculator, and attendees' selections lead to a refined goal and phases focused on features where they disagreed.
Figure 4: (A) Sequence of phases and tool for determining meeting focus. (B) Participants in the meeting employ the focus tool to select preferred features. (C) CoExplorer uses the aggregated preferences to adjust the meeting's objective and flow. From the paper, Fig. 4, p. 6.

How to cite

Gun Woo (Warren) Park, Payod Panda, Lev Tankelevitch, and Sean Rintel. 2024. The CoExplorer Technology Probe: A Generative AI-Powered Adaptive Interface to Support Intentionality in Planning and Running Video Meetings. DIS 2024. https://doi.org/10.1145/3643834.3661507

This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.

BibTeX

@inproceedings{park2024the,
author = {Park, Gun Woo (Warren) and Panda, Payod and Tankelevitch, Lev and Rintel, Sean},
title = {The CoExplorer Technology Probe: A Generative AI-Powered Adaptive Interface to Support Intentionality in Planning and Running Video Meetings},
booktitle = {DIS 2024},
year = {2024},
month = {July},
publisher = {ACM},
url = {https://doi.org/10.1145/3643834.3661507},
doi = {10.1145/3643834.3661507},
}