Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work

Kewenig, Viktor; Tankelevitch, Lev; Rintel, Sean; Sarkar, Advait; Sellen, Abigail

Generative AI imposes new metacognitive demands on knowledge workers as their work shifts from production to planning, evaluation, and orchestration. In a randomized, between-subjects experiment on AI-assisted writing (297 workers, 594 essays), we test dynamically-generated metacognitive scaffolds supporting goal specification and task decomposition. We find that scaffolds improve writing quality, with this boost persisting in short-lag transfer when scaffolds are removed. We also find evidence of improved topic recall three days later. These effects are consistent with observed changes in how people learn to prompt AI systems after exposure to scaffolding. However, scaffolded participants rate the AI as less helpful and more cognitively demanding even as they perform better, suggesting that subjective ease can understate the value of support. Our findings make the case for metacognitive scaffolds in AI-assisted knowledge work, revealing a productive friction worth designing for as workers increasingly delegate work to agentic AI.

Research brief

By Viktor Kewenig, Lev Tankelevitch, Sean Rintel, Advait Sarkar, and Abigail Sellen.

Takeaway: Metacognitive scaffolds (prompts to set goals and break down a task before AI drafts anything) improve people's work and their later prompting. This productive friction feels harder, because people must analyse the task and decide for themselves, but seems to pay off through deeper engagement.

Why this matters

What was known: Interfaces that help people state goals and break tasks down for AI already exist. But they have mostly been judged on how usable they are and how well they steer the model, not on the quality of the work people produce. Yet despite this progress, prior work has primarily examined proximal interaction outcomes, such as usability and model steerability. (§1, p. 2)
What this paper adds: We test whether such scaffolds improve the work itself. We look at output quality while they are available, whether gains carry into a later task without them, and whether people remember their work better. Methodologically, we complement the largely small-scale, within-subjects prior work with a large-scale, between-subjects evaluation designed to isolate scaffolds' causal effects with greater generalizability. (§2.5, p. 6)
Who it matters to: Designers of AI tools for knowledge work, and researchers who evaluate AI's effects on people's thinking, since how a tool feels and how well people perform with it can point in opposite directions. This suggests that studies evaluating AI's impact on cognition, including system evaluations, should include objective measures of performance and cognitive processes (§5.2, p. 21)

Research questions

Do scaffolds improve AI-assisted writing quality, and do effects carry over when working with AI in the absence of scaffolds?
As a mechanism for potential carry-over effects, do scaffolds change how people prompt AI systems in the absence of scaffolds?
Do scaffolds improve people's topic recall of work materials, as a component of cognitive engagement?
How do scaffolds impact perceived helpfulness, sense of control, and cognitive load?

Method and data

Design: Randomized between-subjects experiment. Each participant wrote two AI-assisted professional briefs in counterbalanced order. The treatment group had the scaffold for Task 1 only, and neither group had it for Task 2. Not preregistered.
Prototype: A browser-based writing tool with an AI chat panel and an editable document canvas, connected to GPT-5.3. In the scaffold version, after the user types a high-level request, the system generates a goal panel with editable fields (the message to convey, the target reader, the desired reader outcome, the primary writing objective, and scope and depth). Once these are set, it generates a task-decomposition panel with controls specific to the task (analytical approaches, structure, and critical considerations to address). Users can ask for changes to the controls in plain language, and their selections shape the AI's draft. The controls are generated from the user's own request and earlier choices, not from a fixed template. The control group used the same chat and canvas without the two panels.
Setting: Online and self-paced. Participants wrote two professional briefs: a city proposal on a 4-day work week for municipal employees, and a consulting brief on launching hybrid work, with a bonus for the top five.
Participants / data: 297 US-based Prolific participants with English as their first language (153 scaffold, 144 control), who wrote 594 essays. 159 completed the recall follow-up.
Duration: One self-paced session of up to 80 minutes, plus a follow-up survey three days later.
Analysis: Essays scored on a seven-item rubric by an LLM judge (GPT-5.4) on de-formatted text, checked against two human raters on 117 essays. Mixed-effects models with baseline covariates. Task 2 prompts coded blind for goal clarity and task decomposition (ordinal regression). Topic recall by logistic mixed models with multiple imputation for attrition. Workload by NASA-TLX.

Key findings

Better work, during and after. Knowledge workers who wrote a professional brief with the scaffold produced better briefs than those using a standard AI chat-and-canvas interface, and kept the advantage on a second brief that both groups wrote without it. The scaffold took longer, but time did not explain the gain. #
“Adding log elapsed time left both contrasts unchanged (Table A7), and time on task was not independently associated with quality conditional on treatment”
(§4.1.1, p. 14)
More deliberate prompting carried over. On the second brief, with no scaffold for anyone, people who had used it wrote prompts that stated their goals more clearly and broke the work into steps more often (coded blind to condition). Many people in both groups simply pasted in the task wording as their prompt. #
“56 of 153 previously scaffolded participants (36.6%) and 63 of 144 previously unscaffolded participants (43.8%)”
(§4.2, p. 16)
Better memory of the work. Three days later, people who had used the scaffold were more likely to recall what each brief was about. Both groups had used AI, and only about half the participants returned, so the estimate is imprecise. #
“Our comparison is between two AI-assisted conditions, so it does not contradict evidence that AI impairs retention relative to unaided work.”
(§5.1.3, p. 20)
It felt worse while it worked better. While using the scaffold, people rated the AI as less helpful, felt less in control and reported higher workload, especially for prompting and keeping track of ideas. In the next task, without the scaffold, the difference in workload was small and not statistically reliable. #
“the additional burden was concentrated in coordinating and managing the scaffolded interaction while it was active”
(§4.4.3, p. 18)
People noticed the planning, not the clearer goals. Asked about their own process, people who had used the scaffold said they broke the task into smaller steps, but not that they set clearer goals, although their later prompts improved on both. #
“Decomposition leaves inspectable traces, such as a list of parts, whereas a clearer goal may simply feel like knowing what one wants”
(§5.2, p. 21)
Many handed the AI the whole job. Despite a bonus for the best briefs, 40% of all participants pasted the task description in as their prompt for the second brief, delegating the specification of the work as well as its production. #
“This may partly reflect our cohort, but it is also overreliance on AI in its most complete form”
(§5.3, p. 21)

Questions this paper answers

Does making people plan before they use AI improve their work?: It can, because how clearly goals and task structure are set shapes everything downstream. In a randomized experiment, knowledge workers prompted to state goals and break down a writing task produced better briefs than with a standard chat interface. They kept the advantage on a later task without the prompts. (See §4.1.1, p. 14.)
Can people learn to prompt AI better, or does the tool have to keep helping?: Some of the skill transfers. After one task with a planning scaffold, people's own prompts stated goals more clearly and broke work into steps more often. Whether this lasts for weeks or months has not been tested. (See §4.2, p. 16.)
Why might a tool that improves your work feel less helpful?: Because the effort is visible and the benefit is not. Planning support asks you to analyse the task and make decisions, which raises felt workload, while the better quality shows only in the result. We call this productive friction. (See §4.4.3, p. 18; §5.2, p. 21.)
Does using AI harm people's memory of their work?: Compared with a standard chat interface, a planning scaffold improved people's recall of what they had written about three days later. That does not show AI is harmless to memory compared with working unaided, since everyone compared had used AI. (See §5.1.3, p. 20.)

Quantitative results

Measure	Result	Where
Essay quality (1–4 rubric composite), Task 1 with scaffold vs control	b = 0.089, 95% CI [0.023, 0.155], P = .008	§4.1.1, p. 13
Essay quality, Task 2 (neither group had the scaffold)	b = 0.070, 95% CI [0.004, 0.136], P = .037	§4.1.1, p. 13
Standardized effect size, Task 1 and Task 2	Hedges' g = 0.29 and g = 0.21	§4.1.1, p. 13
Time on Task 1, scaffold vs control (minutes)	27.84 versus 16.35	§4.1.1, p. 14
Human raters' composite advantage on the 107-essay benchmark, Task 1	0.440 [0.157, 0.724], P = .002	Table 2, p. 15
LLM judge agreement with human raters	ICC(A,1) = 0.456, Pearson r = 0.662	§4.1.1, p. 14
Task 2 prompt score, goal clarity + task decomposition (of 39), previously scaffolded vs control	14.43 versus 11.07	§4.2, p. 16
Odds of a higher combined prompt score after prior scaffolding	OR = 1.779 [1.173, 2.698], P = .0067	Table 3, p. 16
Participants whose Task 2 prompts copied the task text	119 of the 297 participants (40%)	§5.3, p. 21
Topic recall three days later, Task 1, scaffold vs control (complete cases)	64.6% versus 45.6%; OR = 5.07, 95% CI [1.21, 21.25], P = .026	§4.3.1, p. 17
Topic recall, Task 2, previously scaffolded vs control (complete cases)	54.4% versus 38.0%; OR = 4.24, 95% CI [1.02, 17.57], P = .046	§4.3.1, p. 17
Topic recall after multiple imputation, Task 1 and Task 2	OR = 3.24, P = .032; OR = 3.06, P = .042	§4.3.2, p. 17
Perceived AI helpfulness (1–5), scaffold vs control	b = -0.342, 95% CI [-0.510, -0.174], P < .001	§4.4.2, p. 18
Perceived control (1–5), scaffold vs control	b = -0.251, 95% CI [-0.486, -0.017], P = .036	§4.4.2, p. 18
NASA-TLX workload (1–10), Task 1, scaffold vs control	M = 4.71 versus 3.96; b = 0.745, P < .001	§4.4.3, p. 18
NASA-TLX workload, Task 2 (no scaffold), not reliable	M = 4.08 versus 3.93; P = .429	§4.4.3, p. 18

Contributions and implications

Treat friction as a design material, but design it to be productive. Stage goal-setting across the draft rather than front-loading it, explain the choices the tool asks for, and pair planning support with support for revision. (§5.3, p. 21)
Make the value of the extra effort visible in the moment, as work moves between phases, and in the outcome, so that people do not abandon a tool that feels harder but works better. A tool that feels less helpful is likely to be declined when it is optional: its benefits arrive only once the task is done (§5.3, p. 21)
A brief scaffold may be a cheap way to build the skill of directing AI, a skill that is unevenly distributed and that people can take from one tool to the next. (§5.1.2, p. 20)
Studies of AI's effects on thinking should measure performance and process objectively, not rely on how helpful or effortful people say a tool felt.

Limitations

Essay quality was scored by an LLM judge whose agreement with human raters was moderate. On a shared subset, human raters found a larger scaffold advantage than the judge did, and the prompting and recall results do not depend on the judge. agreement with human ratings was moreover moderate rather than strong, and the judge compressed the human score range (calibration slope = 0.375; Table A8) (§5.5, p. 22)
The recall result rests on about half the participants, uses a lenient scoring rule, and has wide confidence intervals. Topic recall is a narrow measure of engagement. Follow-up attrition was substantial (50.6% of participants in the recall cohort had no scored recall outcome). (§5.5, p. 23)
We tested short-lag transfer within one session, not persistence over weeks or months, with one-shot online tasks and US-based Prolific adults rather than real workplaces.
There was no static-scaffold condition, and goal and task-decomposition support were tested together rather than separately.

›

Key concepts

Dynamic metacognitive scaffold: Interface controls, generated by AI from the user's own request, that prompt goal specification and then task decomposition before a draft is written. Unlike a template: Importantly, the scaffold was not a fixed prompt template: the options shown were generated and updated from the participant's current request, prior selections and current phase of work. (§3.2.1, p. 8)
Productive friction: Our name for the trade-off we found: support that increases felt effort and dampens perceived helpfulness while improving outcomes (§1, p. 3)
Short-lag transfer: Whether a benefit persists into a later task in the same session once the support is taken away. Our test: in a subsequent writing task in the same session where we remove the scaffolds, we evaluate how scaffold exposure affects written output quality and prompting behavior (§1, pp. 2–3)
Goal clarity and task decomposition: The two dimensions we coded in participants' own prompts, blind to condition: goal clarity (GC), which captures how explicitly participants stated the intended outcome, audience, constraints and success criteria (seven indicators), and task decomposition (TD), which captures whether they divided the work into concrete steps or components (six indicators) (§3.4.2, p. 10)
Cognitive offloading: Delegating production to AI can entail offloading, in the standard definition: recruiting external resources to reduce a task's cognitive demands (§2.2, p. 4)

Key figures

Three panels. The writing interface, with prompt controls for goals and planning beside a chat and document canvas. The study design, with two writing tasks and a recall test 3 days later. A summary of key findings. — A, The AI-assisted writing interface with the metacognitive scaffold: after an initial request (1), dynamically generated controls prompt goal specification (2) and task decomposition (3) before a draft is generated (4); controls could be revised in natural language (5). Figure 1 from Viktor Kewenig, Lev Tankelevitch, Sean Rintel, Advait Sarkar, and Abigail Sellen. 2026. Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work. arXiv preprint arXiv:2610.30945. (Fig. 1, p. 1)

Dot plot of essay quality on a 1 to 4 scale. Scaffolded participants score slightly higher than unscaffolded ones in Task 1, and previously scaffolded participants remain slightly higher in Task 2. — Fig. 3. Essay quality by condition and task. Scores are means across the seven 1-4 rubric ratings. Faint points show observed essay-level scores; larger markers show condition means with 95% confidence intervals. Neither group had the scaffold in Task 2. Figure 3 from Viktor Kewenig, Lev Tankelevitch, Sean Rintel, Advait Sarkar, and Abigail Sellen. 2026. Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work. arXiv preprint arXiv:2610.30945. (Fig. 3, p. 14)

Three dot plots. Scaffolded participants rate AI helpfulness and perceived control lower after Task 1, and report higher NASA-TLX load in Task 1 but not in Task 2. — Fig. 6. Subjective experience by condition. A, AI helpfulness and B, perceived control after Task 1; C, unweighted NASA-TLX load after Tasks 1 and 2 (higher, more load). Figure 6 from Viktor Kewenig, Lev Tankelevitch, Sean Rintel, Advait Sarkar, and Abigail Sellen. 2026. Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work. arXiv preprint arXiv:2610.30945. (Fig. 6, p. 19)

How to cite

Viktor Kewenig, Lev Tankelevitch, Sean Rintel, Advait Sarkar, and Abigail Sellen. 2026. Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work. arXiv preprint arXiv:2610.30945.

This brief was prepared with AI assistance (Claude, by Anthropic) and published by Sean Rintel. It is not the authors' published text. Page numbers refer to this PDF. Quotations and numbers were checked against the full text on 2026-10-25. Machine-readable version: metacognitive-scaffolds-create-productive-friction-for-ai-assisted-knowledge-work.json.