Research overview

Better Thinking With AI

Our papers explore how generative AI moves people's thinking into stating what they want and judging what they get. They suggest that the easiest way to use it can help least, and that good tools ask people to think first.

My colleagues and I published these papers from 2023 to 2026, on how generative AI changes thinking at work and in learning.

They range from a review of automation research and a survey of knowledge workers to randomised experiments, interviews, prototypes, and position papers. Each links to a research brief checked against the paper. Most of the studies observed a single session, an online panel, or one company, so long-term effects at work and in classrooms are not yet known.

Metacognitive demands of generative AI: The metacognitive demands of generative AI are the ways it asks users to monitor and control their own thinking. Users need self-awareness of their goals and must break tasks down to prompt. They need well-adjusted confidence to judge output and metacognitive flexibility to change strategy. They must also decide whether and how to use AI in their work.

How AI changes the thinking people do

What the papers show together

Strongest evidence Also points this way Complicates it

ClaimReviewproductivity loss with AISurveycritical thinking at workConceptual analysismetacognition and AIParticipatory promptingAI-assisted data analysisCommentaryrole-based promptsRandomised trialhow students learn with AIRandomised experimentnote-taking versus AI for readingInterviewsAI in higher educationRandomised experimentscaffolds for AI-assisted writingLab studyprompt refinement controlsWorkshop proposalAI as a tool for thoughtSystem designAI agents for diverse ideas
AI shifts effort from doing work to overseeing itReview: productivity loss with AI, strongest evidenceSurvey: critical thinking at work, strongest evidenceParticipatory prompting: AI-assisted data analysis, also points this way
Effective AI use calls for saying what usually goes unsaidSurvey: critical thinking at work, also points this wayConceptual analysis: metacognition and AI, strongest evidenceParticipatory prompting: AI-assisted data analysis, strongest evidenceCommentary: role-based prompts, also points this wayRandomised experiment: scaffolds for AI-assisted writing, also points this way
People more confident in AI for a task report less critical thinkingReview: productivity loss with AI, also points this waySurvey: critical thinking at work, strongest evidenceConceptual analysis: metacognition and AI, also points this wayCommentary: role-based prompts, also points this wayInterviews: AI in higher education, complicates it
What feels helpful is not always what helpsReview: productivity loss with AI, complicates itRandomised experiment: note-taking versus AI for reading, strongest evidenceRandomised experiment: scaffolds for AI-assisted writing, strongest evidenceLab study: prompt refinement controls, also points this wayWorkshop proposal: AI as a tool for thought, also points this way
How people use AI is linked to what they learn, and productive use is unevenly spreadRandomised trial: how students learn with AI, strongest evidenceRandomised experiment: note-taking versus AI for reading, strongest evidenceInterviews: AI in higher education, also points this wayRandomised experiment: scaffolds for AI-assisted writing, also points this wayWorkshop proposal: AI as a tool for thought, also points this way
Asking for goals before AI drafts can improve the workReview: productivity loss with AI, also points this wayConceptual analysis: metacognition and AI, also points this wayCommentary: role-based prompts, also points this wayRandomised experiment: scaffolds for AI-assisted writing, strongest evidenceLab study: prompt refinement controls, also points this waySystem design: AI agents for diverse ideas, also points this way

AI shifts effort from doing work to overseeing it

As generative AI takes on more of the production of work, our papers explore the tensions this creates for the people who remain responsible for it. Our review of decades of automation research argues that this repeats an old irony, because automation makes human oversight more important even as it takes over the work. In our survey of 319 knowledge workers, people welcomed the effort AI saved them in gathering information and drafting, yet described new work in verifying, integrating, and steering what it made. In that paper, we suggest that stewardship fits this role better than teammate or collaborator, because the person remains accountable for work the AI produced. In our study of 15 spreadsheet users, people valued the chatbot as a partner for thinking, one calling it a rubber duck that talks back, yet found verifying its answers burdensome.

Strongest evidence

  • Review: productivity loss with AI. Our review argues that generative AI repeats the ironies of automation, because checking output people did not create is demanding without the situational awareness that producing it would have given them.
  • Survey: critical thinking at work. Knowledge workers said AI saved effort in gathering information, solving problems, and drafting. They described new effort in verifying information, integrating AI responses, and steering the AI while staying accountable, which we call task stewardship.

Also points this way

Effective AI use calls for saying what usually goes unsaid

We argue that prompting makes metacognitive demands, because goals that stay implicit in manual work must be stated and tasks broken down. In our survey of 319 knowledge workers, 30 described forming or revising prompts as critical thinking, and 45 said turning their intentions into prompts took more effort. In our study of 15 people working with AI on data problems they brought, they struggled to express their intent even with a researcher writing the prompts.

Strongest evidence

Also points this way

People more confident in AI for a task report less critical thinking

In our survey of 319 knowledge workers, people reported less critical thinking when they were more confident in the AI for a task, and more when they were confident in themselves. Overall trust in AI showed no such link, though higher trust went with less reported effort in four of the six critical thinking activities. Of the 319, 55 said they skipped critical thinking when a task seemed trivial, and 44 said they lacked time for it.

Strongest evidence

  • Survey: critical thinking at work. Knowledge workers who were more confident in the AI for a task reported less critical thinking, and those more confident in themselves reported more.

Also points this way

  • Conceptual analysis: metacognition and AI. A conceptual analysis argues that fluent AI output may raise confidence in the output and in people's own judgement of it, so they put less effort into checking.
  • Review: productivity loss with AI. Our review cites studies of complacency and over-reliance on AI in programming, data science, law, and advertising writing, as automation research predicts.
  • Commentary: role-based prompts. I argue that reading AI responses as reflections of how people describe roles and actions could make AI more explainable, helping people calibrate their confidence in their prompts and in AI output.

Complicates it

  • Interviews: AI in higher education. Six of 26 university students we interviewed said domain expertise let them sense-check AI output, and that without it they would trust the AI less or not use it.

What feels helpful is not always what helps

In two experiments, people enjoyed or rated more helpful the easier AI condition while their learning or work gained more from the harder one. Judging tools by satisfaction or speed can pick the wrong design.

Strongest evidence

Also points this way

  • Lab study: prompt refinement controls. Knowledge workers preferred controls generated for each request and rated them more effective for steering, though they felt they had to think slightly harder. We suggest that some added mental load may help by preventing over-reliance on AI.
  • Workshop proposal: AI as a tool for thought. A workshop proposal argues that designs adding friction can meet resistance, and that tools should be judged by how they change thinking, not just output.

Complicates it

  • Review: productivity loss with AI. Our review argues from automation research and early AI studies that AI can make easy tasks easier and hard tasks harder, which it calls task-complexity polarization. Whether the easy route helps may then depend on the task.

How people use AI is linked to what they learn, and productive use is unevenly spread

In our schools experiment, students aged 14 to 15 understood and remembered texts better three days later after taking notes, alone or beside a chatbot, than with the chatbot alone. In our trial with 318 university students, those whose chat logs showed them trying problems first, checking AI responses, or correcting errors scored higher on an exam without AI than students without AI. Students chose these habits, so this is an association. Students from higher-ranked universities or with stronger prior knowledge did this more often. Equal access to AI may therefore not bring equal learning.

Strongest evidence

Also points this way

Asking for goals before AI drafts can improve the work

In one experiment, asking people to state goals and break down the task before AI drafted improved their work. Other designs that offered questions and options and left judgement to people point the same way but were not shown to improve outcomes. The cost is effort, which has to be seen to pay off.

Strongest evidence

Also points this way

  • Conceptual analysis: metacognition and AI. We propose that tools could support people's planning and self-evaluation, and that some added effort may be justified when it improves their thinking and work. Learning research suggests such prompts could fade as people adopt the strategies.
  • Lab study: prompt refinement controls. In our lab study, seven of 16 knowledge workers described controls generated from each prompt as guidance for steering the AI, and five said the controls scaffolded their thinking about the task. Ratings of task success after each tool did not differ significantly.
  • System design: AI agents for diverse ideas. Our YES AND prototype, not yet evaluated, has AI personas speak only when confident they can contribute meaningfully and ask the user clarification questions. Its summary offers only a seed of an approach for the user to develop.
  • Review: productivity loss with AI. A review proposes, from Human Factors research not yet tested with generative AI, that feedback, explanations, and well-timed suggestions could keep people in the loop without breaking concentration.
  • Commentary: role-based prompts. I suggest that systems could coach people to specify the interactional stances that help AI give relevant work information, and later help teams understand one another's stances.

Questions by audience

For researchWhat we still need to study

How do we show that an AI tool protects thinking, not just speeds work?

Studies should assess how a tool changes people's thinking and learning, as well as their output and how the tool felt. In our experiment with 297 workers, people rated the AI less helpful while their briefs improved, so self-reports alone can mislead. Our Tools for Thought workshop proposal suggests process tracing, analysis of what people produce, cognitive and metacognitive assessments, and longitudinal studies to do this. Each of our experiments so far tested a single learning or work session.

From Tools for Thought, Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work

Why do people skip checking AI output, and can design change that?

In our survey of 319 knowledge workers, people described three kinds of barrier to critical thinking with AI. Awareness barriers kept them from seeing a need for it. These arose when AI use was secondary, a task seemed trivial, or they trusted and relied on the AI. Motivation barriers included lack of time and seeing checking as outside their job. Ability barriers included lacking the domain knowledge to inspect a response, or being unable to improve it. All of these barriers come from people's own accounts in the survey. Our analysis of metacognitive demands argues that fast, fluent output may inflate confidence and reduce checking, and that showing model uncertainty could help people adjust their confidence. Experiments are needed that vary feedback on when output is reliable and measure checking directly.

From The Impact of Generative AI on Critical Thinking, The Metacognitive Demands and Opportunities of Generative AI

Who benefits most from AI in learning, and why?

Students from higher-ranked universities or with more prior knowledge more often tried problems first, checked the AI, or corrected it, and this statistically accounted for part of their exam advantage. Signs that they gained more from having AI were not statistically significant. Larger, longer studies that teach these habits are needed to test whether equal access yields unequal learning.

From Learning behavior accounts for background-related advantage in AI-assisted education

For industryResearch-informed practice and what to design next

Should AI writing tools ask people to plan before generating?

Our experiment with 297 workers suggests it can pay off. A tool that prompted people to set goals and break down the task before the AI drafted improved their briefs and their later prompting. While using it, people rated the AI less helpful, felt less in control, and reported more workload. From participants' comments, we suggest spreading goal-setting across the draft instead of asking for it all up front, and showing people how the effort pays off, though workplace use has not been tested.

From Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work

How can tools help people check AI output?

Our review argues that generative AI shifts people's role from production to evaluation, and evaluating output is hard without the context that making it provides. We propose continuous feedback and explainability, such as showing confidence, alternative solutions, and sources. In our study of 15 spreadsheet users working with Bing Chat through a researcher, participants found verifying cited sources burdensome, and one felt it might have been quicker to find the sources themselves. Hallucinations were hard to detect outside participants' expertise. We suggest co-audit tools, where AI helps check AI output, such as generated tests for code and formulas.

From Ironies of Generative AI, "It's like a rubber duck that talks back"

How can tools help people steer AI output?

We built dynamic prompt middleware, in which AI turns a single prompt into a parameterised interface of prompt refinement controls. Each control names one parameter of the request, so the controls scaffold people's thinking about the task. In a study with 16 knowledge workers, these generated controls were rated more effective for steering than fixed ones. Two participants said the controls would help people working in unfamiliar domains, and for four, the controls replaced follow-up prompts during the study. In a direct comparison, participants judged them slightly more demanding, though rated mental load did not differ significantly. With both tools, how the AI would apply an option was often unclear.

From Dynamic Prompt Middleware

For governmentHow this research relates to policy

What should AI literacy cover beyond knowing how to use a tool?

Some governments now require organisations to take steps so that staff who use AI are AI literate. Our studies suggest that literacy should cover how thinking changes, as well as how a tool works. In our survey of 319 knowledge workers, those more confident in AI for a task reported less critical thinking, and those more confident in themselves reported more. They described their effort shifting from gathering information to verifying it, and from doing tasks to overseeing the AI, which we call task stewardship. We argue that using AI well also means stating goals and breaking tasks down before asking for output. In our experiment with 297 workers, a tool that prompted these steps improved their writing, though they rated the AI less helpful and more demanding. The next step is to test which kinds of training build these skills at work.

From The Impact of Generative AI on Critical Thinking, The Metacognitive Demands and Opportunities of Generative AI, Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work

How should schools and universities evaluate generative AI study tools?

Evaluations should test what students retain days later as well as what they prefer. In our preregistered experiment, students aged 14 to 15 in seven English schools studied texts with an AI chatbot, by taking notes, or both. Three days later, chatbot-only study gave lower comprehension and retention than either alternative. Yet 42% of students who compared the chatbot with note-taking preferred it, against 27% who preferred notes. Evaluations should also ask who benefits from a tool and who does not. In our trial with 318 university students, those whose chat logs showed them trying problems first, checking AI responses, or correcting errors scored higher on an exam without AI than students without AI. Students chose these habits, so this is an association. Students from higher-ranked universities or with stronger prior knowledge engaged this way more often. At two UK universities, 17 of the 26 students we interviewed described guidance on AI as unclear, as did 7 of the 11 educators, and students who knew it said its main message was a plagiarism warning. None of these studies has yet tested long-term effects in classrooms.

From Effects of LLM Use and Note-Taking On Reading Comprehension and Memory, Learning behavior accounts for background-related advantage in AI-assisted education, The New Calculator?

What should employers consider when introducing generative AI at work?

Employers introducing generative AI should plan for how it changes the work itself. Our review of automation research and recent AI studies argues that people shift from producing work to evaluating AI output, often with little context. It also argues that prompting and adapting output become new tasks, and that AI can make easy tasks easier and hard tasks harder, which it calls task-complexity polarization. In our survey of 319 knowledge workers, people described moving from doing tasks to overseeing the AI, while staying accountable for the result. The review also suggests making clear which tasks the AI handles and which stay with people. This research cannot say which workplace rules are best. Those are value choices for government, and we welcome a conversation about the evidence.

From Ironies of Generative AI, The Impact of Generative AI on Critical Thinking

What we don't know yet

Open questionWhy it matters
Do metacognitive scaffolds keep helping over weeks of real work?In our experiment with 297 workers, gains from the scaffold carried into a second brief written later in the same online session. Longer use and real workplaces have not been tested.
Does confidence in AI lead people to think less critically, or do people who think less critically rely more on AI?In our survey of 319 knowledge workers, those more confident in AI reported less critical thinking, and those more confident in themselves reported more. A survey cannot show which comes first. Our review of automation research argues that shifting people to checking AI output can breed complacency and over-reliance. Our paper on metacognitive demands argues that fast, fluent output may inflate confidence in one's own judgement. In our experiment with 318 university students, chat logs showed that those with stronger prior knowledge more often tried problems first, checked the AI, or corrected it. None of these studies tested whether changing confidence changes checking.
Does the note-taking advantage hold with newer models and repeated classroom use?Our schools experiment tested one learning session with GPT-3.5 turbo in seven schools in England.
Can teaching AI habits close the gap between better and less prepared learners?In our experiment with 318 university students, those from higher-ranked universities or with stronger prior knowledge more often tried problems first, checked AI responses, or corrected errors. In our writing experiment with 297 workers, a scaffold for setting goals and breaking down tasks improved how people later prompted on their own. That experiment did not test whether the scaffold narrows the gap between strong and weak planners.
How should tools be judged by their effect on thinking as well as output?Our workshop proposal suggests process tracing, analysis of what people produce, cognitive and metacognitive assessments, and longitudinal studies.

The papers