Research brief
Takeaway: Generative AI saves knowledge workers effort but can cost critical engagement. Critical thinking shifts from doing tasks to stewarding them (verifying and integrating AI output). Trusting the AI goes with less reported critical thinking and self-confidence with more, and people skip scrutiny on tasks that seem unimportant.
Why this matters
- What was known
- Research has examined GenAI's effects on memory, creativity and writing, mostly in education, and has used output diversity as a flawed proxy for critical thinking. Its effect on critical thinking in knowledge work had not been studied directly.
For instance, users who reuse GenAI output without editing it may have nonetheless performed a critical, reflective judgement in forming the decision not to edit it.
(§1, p. 2) - What this paper adds
- A survey of a professionally diverse set of knowledge workers, with 936 real-world examples, that models when they perceive themselves enacting critical thinking with GenAI and how GenAI changes its effort.
However, there is a gap in understanding knowledge workers' perceptions of how GenAI affects their enaction of critical thinking, and the effort of doing so, across a broad range of use cases.
(§2.3, p. 4) - Who it matters to
- Designers of GenAI tools for knowledge work, and those training knowledge workers. (§7, p. 16)
Research questions
- RQ1 When and how do knowledge workers perceive the enaction of critical thinking when using GenAI?
- RQ2 When and why do knowledge workers perceive increased/decreased effort for critical thinking due to GenAI?
Method and data
- Design
- Online survey. For each of three real examples of GenAI use at work, participants reported task type, confidence (in self, in GenAI, in evaluating output), whether they enacted critical thinking, and the change in effort for six Bloom's taxonomy activities. Plus reflective thinking, trust in GenAI and demographics.
- Setting
- Prolific. Knowledge workers who self-reported using GenAI tools at work at least once per week.
- Participants / data
- 333 responses received, 14 excluded. 319 participants (159 men, 153 women, 5 non-binary/gender diverse, 2 prefer not to say) sharing 936 retained examples (374 Creation, 303 Information, 259 Advice).
- Duration
- Average 43.19 minutes to complete the survey.
- Analysis
- One random-intercepts logistic regression (perceived enaction) and six random-intercepts linear regressions (perceived effort), with Benjamini-Hochberg correction and adjusted p-values reported. Open coding of free-text responses: a codebook built by one researcher with three others on 50 responses, then two coders coded the remaining 269.
Findings
Critical thinking means quality control. Asked about their own recent uses of generative AI at work, knowledge workers described critical thinking as making sure the work was good. That meant setting clear goals and refining prompts, checking responses against criteria and sources, and adapting responses to fit. #
“I noticed it was missing soap dispensers. So I changed my prompt to include them and tried again...”
Trusting the AI, thinking less. Across their examples, the more confident people were that the AI could do a task, the less likely they were to report thinking critically about it. Confidence in their own ability, confidence in evaluating AI output, and a general habit of reflecting on their work all went with more. #
(§4.2, p. 9)Why people skip checking. People thought critically to improve quality, avoid negative outcomes and build their skills. They skipped it when they did not see the need (the task seemed trivial, or they trusted the AI) or lacked the motivation (no time, or not their job). They also skipped it when they lacked the ability to inspect or improve the output. #
“[t]he reason I use AI is because in sales, I must reach a certain quota daily or risk losing my job. Ergo, I use AI to save time and don't have much room to ponder over the result.”
AI made thinking feel easier. For most examples and most kinds of thinking, from recalling and understanding information to analysing and combining it, people said generative AI took less effort than working without it. Evaluating was the least affected. #
“are immediate and at a sufficient level of detail for me to get to grips with the basics of the industries. I would otherwise have to read a lot of press reports and subscribe to multiple newsletters.”
Self-confident users reported more effort. Confidence in the AI went with less perceived effort, especially in evaluating its output. People confident in their own ability reported more effort in applying and evaluating AI responses. #
“as knowledge workers have more confidence in doing the task themselves, they employ more engaged practices in steering AI responses, especially when applying (Application) and evaluating (Evaluation) AI responses”
From doing to stewarding. Effort moved rather than vanished. It shifted from gathering information to verifying it, from solving problems to fitting AI responses into the work, and from doing tasks to overseeing the AI (which we call task stewardship). #
“the extra effort in determining that the code generated matched my existing code, and making subsequent alterations to make it fit was more effort than just doing it myself in the end.”
Questions this paper answers
- Does using generative AI make people think less critically at work?
- It can. Among 319 knowledge workers using generative AI weekly at work, those more confident in the AI for a task reported less critical thinking, and most felt AI reduced its effort. That points to a risk of overreliance, though these were self-reports, not measured skills. (See §4.2, p. 9; §5.2, p. 12.)
- Does 'less effort' with AI mean people are thinking less?
- Not necessarily. Survey respondents sometimes confused the reduced effort of using AI with reduced effort in critical thinking, especially when satisfied with the output. Telling the two apart needs other measures, such as think-aloud protocols or task-based assessments. (See §5.2, p. 12.)
- Why don't people think critically when using ChatGPT or Copilot?
- Because awareness, motivation or ability is missing. Some do not see the need (a trivial task, a trusted tool). Some have no time or see checking as outside their job. Others lack the domain knowledge to inspect and improve the output. (See Section 4.3.2, p. 11.)
- Does expertise protect against over-relying on AI?
- Possibly. People more confident in their own ability to do a task reported more critical thinking and more effort in evaluating AI output. We cannot show cause and effect, and self-confidence may not match real expertise, but fostering domain expertise may improve critical thinking with AI. (See §4.2, p. 9; §5.1, p. 12.)
Quantitative results
| Measure | Result | Where |
|---|---|---|
| Sample | 319 survey responses; 936 real-world examples | §3, p. 4 |
| Examples where participants reported enacting critical thinking | 555 (59.29%) of 936 | §3.3, p. 6 |
| Confidence in AI → perceived enaction of critical thinking (Benjamini-Hochberg-adjusted p) | β=-0.69, p < 0.001 | §4.2, p. 9 |
| Confidence in self → perceived enaction of critical thinking (Benjamini-Hochberg-adjusted p) | β=0.26, p = 0.026 | §4.2, p. 9 |
| Confidence in evaluating AI responses → perceived enaction of critical thinking (Benjamini-Hochberg-adjusted p) | β=0.31, p = 0.046 | §4.2, p. 9 |
| Tendency to reflect on work → perceived enaction of critical thinking (Benjamini-Hochberg-adjusted p) | β=0.52, p < 0.001 | §4.2, p. 9 |
| Examples rated 'much less' or 'less' effort with GenAI, by activity | 72% Knowledge, 79% Comprehension, 69% Application, 72% Analysis, 76% Synthesis, 55% Evaluation | §5.1, p. 12 |
| Confidence in AI → perceived effort in Evaluation (Benjamini-Hochberg-adjusted p) | β=-0.23, p < 0.001 | §5.1, p. 12 |
Contributions and implications
- Design interventions should enhance motivators and mitigate inhibitors of critical thinking.
critical thinking interventions for GenAI tools should aim to enhance and leverage motivators while mitigating and avoiding inhibitors
(§6.1, p. 14) - Help users recalibrate task confidence, with feedback on when AI outputs are reliable and explicit controls over the extent of AI assistance.
- Offer proactive and reactive critical thinking support, frame it as long-term skill development, and help users learn to inspect and guide AI output.
- Train knowledge workers in information verification, response integration and task stewardship, while maintaining foundational skills.
training knowledge workers to think critically when working with GenAI should focus on developing skills in information verification, response integration, and task stewardship
(§6.2, p. 15)
Limitations
- Participants sometimes conflated reduced effort in using GenAI with reduced effort in critical thinking, so self-reports may be inaccurate.
participants occasionally conflated reduced effort in using GenAI with reduced effort in critical thinking with GenAI
(§6.3, p. 15) - The analysis does not establish causation, and self-confidence is subjective and may not match objective expertise.
Our analysis does not establish causation.
(§6.1, p. 14) - The survey was English-only and the sample skewed towards younger, more technologically skilled, regular GenAI users.
- GenAI tools and their use are evolving quickly, and the coarse task taxonomy may miss task-specific factors.
›Concepts
- Critical thinking (Bloom's taxonomy)
- We adopt Bloom and colleagues' six cognitive activities: knowledge, comprehension, application, analysis, synthesis and evaluation.
a hierarchical taxonomy that characterises student learning objectives into six types
(§2.1, p. 3) - Enaction of critical thinking
- Actions that signal or manifest critical thinking, which people can report more easily than the mental phenomenon itself.
We focus on "enaction" (i.e., actions that are signals or manifestations) of critical thinking rather than critical thinking per se
(§1, p. 2) - Overreliance
- Closely related to, but not the same as, a lack of critical thinking:
users accepting incorrect recommendations, i.e., making errors of commission
(§2.2, p. 3) - Critical integration
- Sarkar's term, which our participants' practices bear out:
the work of editing and incorporating GenAI output into a broader workflow
(§4.1, p. 9) - Task stewardship
- Our term for the shift from doing a task to guiding and monitoring AI to produce high-quality outputs, while the human keeps responsibility and accountability.
the responsibility and accountability for the work still resides with the human user despite the labour of material production being delegated to the GenAI tool
(§6.2, p. 15)
Figures
How to cite
Hao-Ping (Hank) Lee, Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson. 2025. The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. CHI 2025. https://doi.org/10.1145/3706598.3713778
This brief was prepared with AI assistance and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF.