YES AND: A Generative AI Multi-Agent Framework for Enhancing Diversity of Thought in Individual Ideation for Problem-Solving Through Confidence-Based Agent Turn-Taking

Authors: Pratik Ghosh, Sean Rintel
Venue: CHI EA 2025
Free from the publisher

AThe paper

A design paper presenting YES AND, a ChatGPT-based chat app in which AI expert personas brainstorm with one person. Two versions were built and compared informally on example conversations, with no user study.

  1. Fixed order versionIn the first version, the user's problem passed through the agents in a fixed sequence.
  2. Turn-taking versionIn the second version, the agent most confident of adding something speaks next.
  3. Example conversationsFour example chat logs from the second version and one from the first, on problems such as recalling sent emails.
  4. DiscussionThe authors discussed excerpts, comparing the fixed order and turn-taking versions.

BThe tool

  1. State a problemThe user types a problem, and three expert personas reply as in a group chat.DesignerMachine Learning ResearcherEngineer
  2. Rate confidenceFor every message, each agent rates from 0 to 10 its confidence that it can contribute.
  3. Most confident speaksThe most confident agent speaks if it clears a threshold of 7. The others just say hmm.
  4. Ask or pass onThe speaker may ask the user a clarifying question or nominate another agent to build on its idea.
  5. Cut in or sum upThe user can interject at any time, or call a Sage agent to summarise a seed of an approach.

Steps 2 to 4 are the confidence-based turn-taking. In the first version, agents answered in a fixed order.

CWhat the paper reports

Fixed orderthe first version

  • In the first version, ideas did not develop spontaneously and the user had little say.The user's problem passed from agent to agent in a fixed order. It worked in principle.
  • The authors observed that language models do not know on their own when to stay quiet.Told that a button placement follows Fitts' Law, the chatbot explained the law back at length.

Confidence-based turnsthe second version, in example conversations

  • Asked to rate its confidence, an agent scored the same statement low and said only hmm.In the authors' testing, such scores worked for choosing who should speak, though models' self-assessment is not fully precise.
  • In the example conversations, agents asked the user clarifying questions before offering solutions.Agents also built on and critiqued each other's ideas.
  • Agent conversations could run on, so the user needed to be able to cut in.The user could interject to redirect the agents at any time, or call the summarising agent to conclude.
  • The paper has not tested whether YES AND improves creativity or engagement.Controlled experiments and user studies are future work. The personas are fixed professional roles that may not fit every problem.

DKey takeaway

AI expert personas may bring group diversity of thought to one person's brainstorm. Personas speaking only when confident of adding something new may keep ideas building and the person engaged.

Made for this site with AI assistance from the verified research brief. It is not the paper's own figure.

Abstract

Diversity of thought is crucial in ideation for problem-solving, yet professionals in organisational settings often face challenges such as limited access to varied expertise and resource constraints which hinder the ideation process. To address this issue, we propose YES AND, a Generative AI based multi-agent framework that simulates diverse perspectives through AI agents for ideation with a single user. Leveraging a unique confidence-based turn-taking model, these agents organically take turns as they build on ideas, pose clarification questions to the user for improved contextual understanding, and allow the user to interject and steer the conversation. Beyond addressing the limitations of traditional ideation, this framework offers a novel approach to leveraging Generative AI for ideation, moving away from the rigidity of pre-defined interaction rules towards a more dynamic and creative process that enables serendipitous development of ideas.

Research brief

By Pratik Ghosh and Sean Rintel.

Takeaway: Working alone, people can lack the diversity of thought a group brings. AI expert personas could fill that gap, brainstorming with one person by borrowing turn-taking from conversation analysis and 'yes, and' from improv. In the YES AND prototype, each persona speaks only when confident of adding something new.

Why this matters

What was known
Diverse ideation leads to more creative outcomes, but individuals often cannot reach diverse colleagues, and existing AI ideation tools give long, definitive answers. most existing multi-agent frameworks for ideation are either limited to specific domains or rely on pre-defined rules for agent interaction which can constrain creativity and serendipitous development of an idea (§2.3, p. 2)
What this paper adds
Instead of fixed interaction rules or long, complete answers, a deliberately simple framework in which agents decide for themselves when to speak and the user is left to develop the solution. It is built through prompt engineering rather than model training. Our framework addresses these limitations by introducing a flexible dynamic multi-agent environment where agents autonomously generate and build on ideas while maintaining alignment with user objectives. (§2.3, p. 3)
Who it matters to
Professionals solving problems in organisations without ready access to diverse expertise, and, the paper suggests, ideation in healthcare, education and finance. promote organisational efficiency and skill-building in individuals (p. 1)

Research question

Method and data

Design
Iterative system design with two versions: V1 passed the user's problem through the agents in a fixed sequence, and V2 used confidence-based turn-taking. Agent behaviour was shaped by prompt engineering in system messages, refined through trial and error.
Prototype
YES AND, a web chat app built with the Gradio Python library on a ChatGPT-based large language model. The user types a problem. Three agents with professional personas (a Designer, a Machine Learning Researcher and an Engineer) reply as in a group conversation, each speaker's picture shown beside the chat. For every message, each agent rates from 0 to 10 its confidence that it can contribute. The most confident speaks if it clears a threshold of 7, and may ask the user a clarifying question or nominate another agent to build on its idea, while the others just say 'hmm'. The user can interject at any time or call a fourth 'Sage' agent to summarise the discussion as a seed of an approach rather than a full solution. Compared informally with the earlier version, in which agents answered in a fixed order.
Flowchart. The user's input is evaluated for topic, intent and nomination, and a nominated agent replies. Otherwise Designer, ML Researcher and Engineer score confidence and the top scorer speaks if at least 7. Failing that, the Sage summarises.
Figure 1: Conversation turn-taking architecture. From the paper, Fig. 1, p. 4.
Setting
Example scenarios of a single user ideating on organisational problems, such as a Program Manager designing a feature to recall sent emails.
Participants / data
Four agents. Four example V2 chat logs and one V1 log are given in the appendix. No user study.
Analysis
Qualitative discussion of excerpts from the example conversations, comparing the linear and turn-taking versions.

Findings

Questions this paper answers

Can a panel of AI personas stand in for diverse colleagues?
Potentially, for people without ready access to diverse expertise. In example conversations, agents with different professional roles built on and critiqued each other's ideas and asked the user clarifying questions. Whether this improves creativity, and how well preset roles fit real problems, is untested. (See §3.4, p. 5; §3.4, pp. 5–6.)
How do you stop multiple AI agents from all talking at once or rambling?
Make each agent judge whether it has something worth adding. Having agents rate their own confidence, and letting only the most confident speak, curbed verbose, off-point replies in testing. Self-assessment is imperfect, though, and the user still needs a way to cut in. (See §3.3, p. 4; §3.4, p. 5; §3.4, pp. 5–6.)
Is a free-flowing conversation better than having AI agents answer in a fixed order?
It seems so. A fixed order restricts the spontaneous development of ideas and gives the user little say. In example conversations, letting agents speak when confident made ideation more interactive and iterative. Agents asked clarifying questions and built on each other's ideas. No user study has tested this. (See §3.2, p. 3; §3.4, p. 5.)

Contributions and implications

Limitations

›

Concepts

Diversity of thought
How we use the term: this work defines diversity of thought as the variation in perspectives and opinions arising from differences in backgrounds and expertise (§2, p. 2)
Design improv ('yes, and')
The inspiration for the framework: accept others' ideas and build on them. Inspired by "design improv" [19], which promotes acceptance of ideas and building upon them (p. 1)
Conversational turn-taking
We adopt the turn-taking model of Sacks and colleagues from conversation analysis: a speaker nominates the next speaker, others self-select, or the speaker continues. (§3.3, p. 3)
Confidence mechanism
Each agent rates its confidence in a message's topic and intent from 0 to 10 and speaks only above a threshold. Otherwise it just says 'hmm'. This approach aimed to limit their responses to only situations where they believed they could confidently and meaningfully contribute to an ongoing conversation. (§3.4, p. 4)
Persona-based agents
AI agents given a professional role (Designer, Machine Learning Researcher, Engineer) through prompt engineering in system messages, plus a 'Sage' that summarises. we focused only on professional role-based personas with a limited set of traits relevant to problem-solving (§3.1, p. 3)

Figures

Flowchart. The user's input is evaluated for topic, intent and nomination, and a nominated agent replies. Otherwise Designer, ML Researcher and Engineer score confidence and the top scorer speaks if at least 7. Failing that, the Sage summarises.
Figure 1: Conversation turn-taking architecture. From the paper, Fig. 1, p. 4.

How to cite

Pratik Ghosh and Sean Rintel. 2025. YES AND: A Generative AI Multi-Agent Framework for Enhancing Diversity of Thought in Individual Ideation for Problem-Solving Through Confidence-Based Agent Turn-Taking. CHI EA 2025. https://doi.org/10.1145/3706599.3720142

This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.

BibTeX

@inproceedings{ghosh2025yes,
author = {Ghosh, Pratik and Rintel, Sean},
title = {YES AND: A Generative AI Multi-Agent Framework for Enhancing Diversity of Thought in Individual Ideation for Problem-Solving Through Confidence-Based Agent Turn-Taking},
booktitle = {CHI EA 2025},
year = {2025},
month = {April},
publisher = {ACM},
url = {https://doi.org/10.1145/3706599.3720142},
doi = {10.1145/3706599.3720142},
}