While writing Cognitive Endurance, I kept seeing studies and headlines about AI weakening memory, learning, and critical thinking. I wanted to understand what researchers were investigating, how those abilities were being measured, and which questions were still open. The field is moving quickly, so I also wanted a baseline I could return to as the evidence changes.
This map brings together 110 source records found through Claude Science and Perplexity. I checked each source against the question it appears under and kept the study design, sample, timing, and limits visible. The search was exploratory and did not follow a systematic-review protocol.
AI studies · earlier technology · cognitive psychology
89 / 21
primary empirical records · other evidence roles
95
peer-reviewed records
53
records coded as randomized
16
records found by both searches
111 source appearances shown
The memory evidence answers seven narrower questions. It shows that AI can distort recall in a suggestive interview, that offloading can trade later memory for immediate performance, and that delayed learning results vary by intervention; it does not establish a general long-term effect of everyday generative-AI use on memory.
Watching a crime video followed by generative chatbot interaction caused over three times more immediate false memories than control, with 36.4% of responses misled; these persisted one week later.
Researchers had people go through a simulated witness interview using a generative chatbot, a pre-scripted chatbot, a survey, or no interview (control), then checked for false memories, memories of things that didn't happen. The generative chatbot produced over 3 times more immediate false memories than the control and 1.7 times more than the survey, and it misled 36.4% of users' responses during the conversation. A week later, the number of false memories from the generative chatbot stayed the same, and people's confidence in these false memories stayed higher than in the control group, suggesting the effect didn't fade over time.
N=200 participants Phase 1 (50 per condition); Phase 2 N=194 (control 49, survey 48, pre-scripted 48, generative 49); 39 additional participants excluded for failing attention checks in Phase 1 survey
Population
Recruited via Prolific online recruitment platform; prescreened to be fluent in English and aged 18 and above; adults; country not reported in retrieved text beyond "online pool"
Measured
Number of false memories induced (count of critical misleading questions answered >4 on 7-point Definitely No–Definitely Yes follow-up scale) measured immediately and after 1 week; confidence in false memories (0–3 scale derived from follow-up scale) measured immediately and after 1 week; confidence in true memories (non-critical questions); emotional state via Self-Assessment Manikin (SAM) scale; cognitive workload…
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
Future research could explore the boundaries of selective memory manipulation with more complex or interconnected memories; longitudinal studies examining persistence of AI-induced false memories beyond one week would provide valuable insights into durability of effects; the study notes prior AI familiarity vs chatbot-specific familiarity distinction may confound interpretation of moderating factors.
Among 28 participants, default and guided GPT produced nearly identical factual-recall scores after a robbery video. In the default group, feeling clearer tracked greater trust in GPT without tracking recall accuracy; in the guided group, felt clarity and recall accuracy aligned. There was no non-GPT control.
Default and guided GPT groups had nearly identical factual-recall scores. In the default group, subjective clarity tracked greater trust in GPT but not recall accuracy; in the guided group, subjective clarity aligned with recall accuracy
Design
Randomized between-groups experiment comparing default GPT with guided GPT after a robbery video
Sample
28 participants
Population
participants who watched a short robbery video (approximating one-time eyewitness scenario); country/setting/adult status not reported in retrieved text
Measured
subjective clarity of understanding of the event, trust in GPT's output, actual recall (alignment with subjective clarity), evaluations of legitimacy of individuals in the incident, and interaction strategies used with GPT (qualitative interaction analysis); no specific instrument named in retrieved text
Timing
with AI present or not reported
Found by
Claude Science
Source basis
full text
Limit
Small sample; no non-GPT control; the study does not establish lasting effects
Among 54 adults writing essays across three sessions, the LLM group showed the lowest task-time EEG connectivity; immediately afterward, 15 of 18 could not accurately quote their essay, versus 2 of 18 in each comparison group. The preprint does not establish lasting brain or learning change.
Kosmyna and colleagues measured brain connectivity, or how strongly different brain regions' electrical signals sync up, while people wrote essays either alone, with a search engine, or with an LLM (large language model, like ChatGPT). The more outside help people used, the weaker and narrower their brain connectivity became: unaided writers showed the strongest, widest networks, search-engine users were in between, and LLM users showed the weakest coupling overall. In a fourth session, people who had used an LLM first and then switched to writing alone still showed weaker connectivity and under-used alpha and beta brain-wave networks, while people who switched from writing alone to using an LLM recalled more and re-engaged widespread brain regions at the back and front of the head. Behaviorally, 83.3% of the LLM group (15 of 18) couldn't correctly quote from the essay they had just written, compared with only 11.1% (2 of 18) in each of the other two groups.
Design
Between-subjects experiment (3 arms: LLM, Search Engine, Brain-only) with repeated sessions (3 sessions, all participants; optional 4th session with cross-over reassignment: LLM-to-Brain and Brain-to-LLM). Not stated as preregistered.
Sample
Total N=60 recruited, 55 completed ≥3 sessions, 54 analyzed (18 per group for Sessions 1-3); 18 participants completed Session 4 (subset reassigned to LLM-to-Brain and Brain-to-LLM subgroups).
Population
Adults aged 18-39 (M=22.9, SD=1.69), recruited from 5 universities in the greater Boston area (MIT, Wellesley, Harvard, Tufts, Northeastern), USA; mix of undergraduate and postgraduate students plus some post-docs/research staff.
Measured
EEG (Enobio 32-channel headset, 500Hz) analyzed via dynamic Directed Transfer Function (dDTF) for neural connectivity across alpha, beta, theta, delta bands; NLP analysis of essays (Named Entity Recognition, n-grams, ontology/topic analysis, latent space embeddings, similarity/distance metrics); essay scoring by human teachers and an AI judge; post-assessment structured interviews assessing essay ownership, ability…
Timing
after AI was removed
Found by
Both searches
Source basis
full text
Limit
Text notes this is a preprint "under review"; authors direct readers to a dedicated "Limitations and Future Work" section which is not included in the supplied excerpt. Supplied text itself states no explicit limitations content beyond noting EOG data were recorded but excluded from the manuscript, and that Session 4 had much smaller N (18) with no Search Engine group in Session 4.; a later methodological preprint (arXiv:2601.00856) raises design, reproducibility, EEG-analysis, reporting, and transparency concerns
Among 85 completers, students who prepared with ChatGPT scored 57.5% on a 20-item retention test 45 days later, versus 68.5% after traditional study; attrition and the single-site design limit the result.
Among 85 completers, retention was 57.5% with ChatGPT versus 68.5% with traditional study, t(83)=-3.19, p=.002, d=0.68
Design
Randomized controlled trial; two-week presentation preparation with unrestricted ChatGPT or non-AI resources; surprise test about 45 days later
Sample
120 randomized; 85 completed follow-up
Population
Undergraduate business students at one Brazilian university
Measured
Study-specific 20-item multiple-choice retention test
Timing
after AI was removed with 45-day follow-up
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Nearly 30% did not complete follow-up; one institution; comparison-group AI non-use was not directly monitored; groups also differed in reported study time
One week after a 35-minute learning task, students allowed to use AI scored 5.1 percentage points higher on an unaided test; both groups declined, and only 68% of the AI group used it.
At one week, randomized AI access raised the unaided knowledge-test score by 5.1 percentage points (0.27 SD), p=.027; both groups declined
Design
Randomized in-person experiment; AI allowed or forbidden during a fixed 35-minute learning task; unaided tests immediately and about one week later
Sample
211 in Session 1; 204 in Session 2
Population
Middlebury College undergraduates
Measured
Unaided factual and conceptual knowledge tests immediately and about one week later
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Working paper at one selective college; 68% of the allowed group used AI; augmentation-versus-automation comparisons were observational
A 32-student pilot found no immediate difference on a 20-item suctioning-knowledge questionnaire. Six weeks later, median scores were 65 with guided ChatGPT plus peer discussion and instructor correction versus 55 with traditional teaching; the bundle prevents attributing the difference to ChatGPT.
No between-group difference immediately (Z=-1.260, p=.208); at six weeks the intervention median was 65 versus 55 in control, Z=-2.014, p=.044
Design
Pilot parallel-group RCT; guided ChatGPT plus peer discussion, guideline verification and instructor correction versus traditional teaching; six-week remote follow-up
Sample
34 randomized; 32 analyzed
Population
Undergraduate nursing students at one Turkish university
Measured
Validated 20-item suctioning knowledge questionnaire administered before immediately after and six weeks after instruction
Timing
after AI was removed with six-week follow-up
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Small underpowered volunteer pilot; bundled intervention; repeated test; unsupervised remote follow-up; authors call the result preliminary
People who used Google for difficult questions were more likely to use it for easier questions later; the experiments measured later search choice rather than memory decline.
Participants who had used Google for difficult questions were more likely to use it for later easier questions; this measures later search choice rather than memory decline
Design
Three experiments
Sample
60 / 80 / 40 participants
Population
University of California Santa Cruz undergraduates
Measured
Choice to use Google or memory for later trivia questions
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Laboratory trivia tasks with undergraduate samples; the study does not test durable memory change
A preregistered replication found no conclusive evidence that difficult questions automatically activate thoughts of internet access, weakening one proposed mechanism behind later search use.
The replication found no conclusive evidence that difficult questions automatically activate the concept of internet access
Design
Preregistered replication
Sample
117 tested; 89 analyzed
Population
German adults
Measured
Stroop interference for internet-related words after difficult versus easy general-knowledge questions
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Failed replication of one proposed mechanism; it does not show that search has no other effect on memory or behavior
Compared with map users and people who learned the route directly, GPS users traveled farther and more slowly, made larger direction errors, and drew poorer sketch maps.
GPS users traveled farther and more slowly, stopped more often, made larger direction errors and produced poorer sketch maps than the map and direct-experience groups
Design
Three-condition comparison of GPS navigation map use and direct route experience
Sample
66 college students
Population
College students
Measured
Travel behavior direction estimates and sketch-map topological accuracy
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
One route-learning task and student sample; a comparison with earlier navigation technology rather than generative AI
Greater lifetime GPS use was associated with poorer unaided spatial memory; the longitudinal follow-up included only 13 people, so causal interpretation remains limited.
Greater lifetime GPS use was associated with worse unaided spatial memory; in the 13-person follow-up more GPS use was associated with steeper decline
Design
Cross-sectional study plus three-year follow-up
Sample
50 regular drivers; longitudinal follow-up n=13
Population
Montreal-area drivers
Measured
Unaided spatial-memory performance and reported lifetime GPS use
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Most evidence is cross-sectional and the longitudinal sample is only 13; association does not establish causation
Taxi drivers with greater navigation-aid dependence performed worse on landmark and survey knowledge and showed greater cognitive load, while route recognition and visual attention did not differ.
Drivers with greater navigation-aid dependence performed worse on landmark and survey-knowledge measures and showed greater cognitive load; route recognition and visual-attention measures did not differ
Design
Between-participants comparison of drivers with high and low navigation-aid dependence; EEG eye tracking and spatial-knowledge tasks
Sample
24 experienced male taxi drivers
Population
Beijing taxi drivers
Measured
Scene and route recognition sketch maps EEG and eye tracking
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Small all-male expert sample; dependence groups were observational rather than randomized; some EEG and eye-tracking analyses had fewer than 24 participants
When people expected information to remain searchable, they remembered less of the information itself and more about where to find it.
Owing to Internet search, we are more likely to encode "where" aspects of memory rather than "what."
Design
4 experiments
Sample
Not reported in the retrieved record
Population
Not reported in the retrieved record
Measured
Not reported in the retrieved record
Timing
with AI present or not reported
Found by
Both searches
Source basis
abstract only
Limit
abstract-only excerpt, no methods provided; single sentence of text available; outcome is proxy not learning; no sample size stated; no population/setting stated
A meta-analysis reported a moderate overall Google-effect estimate, but the included studies measured substantially different constructs, so it is not one estimate of memory loss.
The authors report a statistically significant moderate pooled effect, but the included constructs and individual effects were highly heterogeneous
Design
Meta-analysis
Sample
22 articles; 35 comparisons; total N=30,889
Population
Participants aged 12 to 89
Measured
Heterogeneous outcomes including memory cognitive load behavioral measures and cognitive self-esteem
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
The retrieved main text does not state a single pooled effect size; the synthesis should not be read as one clean estimate of memory loss
Mechanism evidence that offloading can improve immediate performance while reducing later memory; test expectation mitigates the cost.
More offloading improved immediate performance but reduced later memory; expecting a memory test mitigated the memory cost and nearly counteracted it under forced maximal offloading
Design
Three preregistered experiments using a pattern-copy task
Sample
172 participants in each experiment
Population
Adults
Measured
Immediate task performance and later memory for copied patterns
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Laboratory pattern-copy tasks test a mechanism rather than long-term everyday technology use
People systematically set more reminders than optimal, driven by metacognitive beliefs rather than actual ability. Reminders work well, but older adults don't fully compensate for memory decline.
People forgot about 45% of the time when they relied on their own memory, but only about 5% of the time when they used an external reminder. Whether someone chose to set a reminder was tied to their beliefs about how good their memory is, not to how good their memory actually was (r = -0.01) — a mismatch the authors link to a broader tendency to lean on reminders based on gut feelings about memory, wanting to avoid effort, and sticking with familiar habits, rather than to any causal harm or benefit from using reminders. This pattern shifts with age: children under about 9 know they might forget but don't yet act on that by setting reminders, while older adults tend to be overconfident and under-use reminders and younger adults tend to be underconfident and over-rely on them. The authors note these age-related patterns come from studies comparing different people at one point in time, not from tracking the same people over years, so they're less certain.
Design
Narrative review synthesising multiple experimental/behavioural, developmental, ageing, and fMRI studies of intention offloading; not a single primary study
Sample
not reported in retrieved text (pooled synthesis of multiple studies, each with own N; no aggregate k or total N given in supplied text)
Population
Multiple populations across cited studies: adults (younger 18–30 years and older 65–84 years, e.g. Scarampi & Gilbert), children aged approximately 7–13 years (Redshaw et al.) and children as young as 4 years (Bulley et al.) — mixed ages incl. minors across the body of reviewed work; individual…
Measured
Delayed intention/prospective memory performance via a computerised drag-and-circle intention offloading task (Gilbert paradigm); rate/proportion of reminder-setting; accuracy under forced internal-memory vs. forced-reminder trials; optimal reminders paradigm comparing actual vs. optimal choice behaviour; metacognitive confidence/prediction ratings; naturalistic prospective memory task (collecting web-link bonuses…
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
Authors note: "the studies reviewed above were mostly conducted with rather artificial experimental tasks"; "we do not yet know much about how individuals' cognitive offloading strategies, measured in the laboratory, relate to their use of cognitive offloading in everyday life"; findings on ageing/development are cross-sectional and the authors state "this conclusion would be stronger... if it were supported by longitudinal rather than cross-sectional evidence"; effects of metacognitive interventions on cognitive offloading "have been inconsistent" between studies; "the cross-domain organisation of metacognition…
Questions still unanswered
What happens to memory after ordinary, repeated generative-AI use over months or years?
If recall weakens after sustained reliance, does it recover after people stop using the tool?
Which AI interaction patterns create false memories outside a deliberately suggestive eyewitness interview?
How do age, baseline memory, expertise and cognitive impairment change the effect?
Do patterns observed with search and GPS generalize to generative AI?
Which parts of the delayed results come from encoding depth, retrieval practice, time on task, feedback, or the expectation that material will need to be remembered?
The learning evidence supports short-term gains from well-designed AI tutoring and shows that workflow, prior ability, and whether the AI supplies answers or supports thinking can change the result. Most studies stop at an immediate test, so the branch cannot say whether the gains last.
ChatGPT's simulated patient conversations with feedback improved medical students' clinical decision-making compared to conversations without feedback, particularly for gathering patient context and information.
Medical students who practiced AI-simulated patient conversations and then got ChatGPT feedback made better clinical decisions than students who did the conversations alone, scoring 3.604 on average versus 3.017 for the no-feedback group, F(1,20)=4.436, p=.049. Both groups improved steadily from the first to the fourth practice scenario. The benefit showed up only in two specific skills — "creating context" (p=.046) and "securing information" (p=.018) — but not in "focusing questions," or asking targeted follow-up questions, (p=.265), which the authors suggest might be because students were already scoring near the top on that skill (a possible ceiling effect).
Design
Two-group randomized controlled trial (closed-label, participants blinded); repeated measures over 4 scenarios; not stated as preregistered
Sample
Total N=21 analyzed (24 randomized, 29 initially agreed, 5 dropped before randomization, 3 excluded post-randomization for technical/data issues); feedback group n=10, control group n=11
Population
Medical students at the University of Münster, Germany, recruited Nov 2023–Feb 2024; ages 19–35 (adults); none had clinical rotations, half had no prior knowledge
Measured
Clinical decision-making/clinical reasoning assessed via the Clinical Reasoning Indicator-History Taking Inventory (CRI-HTI), an 8-item, 5-point Likert scale across three competence domains (Focusing questions, Creating context, Securing information), rated by two blinded independent human raters from chat transcripts (ICC=0.924); also a post-training survey on perceived realism, authenticity, accuracy, and…
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
ChatGPT 3.5's training data only extends to January 2022; ongoing model updates (e.g., March 2023 update) may affect reproducibility and it has not been tested whether results replicate with newer versions; conversations were conducted in German, and answers/feedback may be more precise in English since ChatGPT was originally trained on English texts; the sample is relatively small though a significant effect was found, and future studies should increase statistical power and participant numbers and examine effects across different geographic/cultural audiences; some participants learned to combine multiple…
ChatGPT hints produced a 17-percentage-point pre/post gain and outperformed no hints; their gains were statistically indistinguishable from human-authored hints.
Students who got math hints written by ChatGPT improved their test scores by 17.00 percentage points, from 43.51% correct before to 60.52% after, a gain much bigger than the 1.85% improvement seen with no hints at all (that small change could have been chance, p=0.192). A statistical test comparing all three groups found a real difference between them, and a follow-up comparison confirmed ChatGPT hints beat no hints. Students who got hints from a human tutor improved by 11.62%, a result close enough to ChatGPT's that the two weren't statistically distinguishable — suggesting AI hints might work about as well as human-tutor hints. But there's a catch: the ChatGPT hints were actually wrong, either in their work or final answer, 32% of the time overall, and up to 47% of the time for one topic (Intermediate Algebra); a fix that made the AI check its own consistency nearly eliminated wrong answers for algebra but still left 13% wrong for statistics.
Design
Randomised controlled experiment, 3x4 design (3 hint conditions x 4 subjects), online, random assignment via Qualtrics; not stated as preregistered
Sample
Total N = 274 participants after exclusions (Control n=90, human-tutor hints n=86, ChatGPT hints n=98); recruited 394, 120 excluded. RQ1 hint-quality analysis unit: 75 problems (questions) ChatGPT was prompted to generate hints for, rated by 6 undergraduate…
Population
Mechanical Turk workers (US-based crowdsourcing platform, online); adults — required at least a high school degree and MTurk "Master" designation; median age group of participants was 36-40
Measured
Learning gain measured via a three-item pre-test and identical three-item post-test per subject (algebra/statistics problems from OpenStax textbooks); also measured session/time-on-task; hint quality measured via 3-point quality check (correct answer, correct work, no inappropriate language) rated by 6 undergraduate raters with Fleiss' Kappa for inter-rater reliability
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
Hints could only be produced for problems without graphical figures due to ChatGPT's text-only input/output limitation at the time; study relied on a closed-source LLM whose model weights are not public; used crowdsourced (Mechanical Turk) learners rather than in-situ secondary/post-secondary students, though authors cite findings that crowdsourced results agree with in-situ learners; high attrition rate of 30%, though fairly even across conditions; scope limited to secondary and early post-secondary mathematics.
An AI tutor using GPT-4 with step-by-step guidance produced substantially larger learning gains than in-class active learning, with students also reporting higher engagement and motivation.
Students using an expert-designed AI tutor had higher post-test scores and more than double the median learning gains of the in-class active-learning condition. The crossover study included 194 eligible students, and the reported group difference was p < 10^-8. The result applies to a tutor built with GPT-4, expert prompts, instructional videos, and prewritten solutions.
Design
Randomised controlled trial in an authentic course setting; AI tutor vs in-class active learning, two lessons over two consecutive weeks
Sample
Total N=194 eligible (of 233 enrolled); post-test analysis: AI group N=142, in-class group N=174; combined pre-test N=316 (crossover design so students contribute to both conditions)
Population
Undergraduate students in Physical Sciences 2 (PS2), an introductory physics course for life sciences at Harvard University, USA; adults (undergraduate university students)
Measured
Content mastery via pre-test and post-test quizzes on lesson-specific physics content (surface tension, fluid flow); prior physics background via midterm exam and Force Concept Inventory (FCI); scientific attitudes via CLASS survey; time on task (AI platform tracked); student perceptions of learning experience via four Likert-scale (1-5) statements on engagement, motivation, enjoyment, and growth mindset
Timing
after AI was removed
Found by
Claude Science
Source basis
full text (PMC12179260)
Limit
Authors state: the AI tutoring approach was applied in a setting where students engaged substantially with material for the first time, with lessons at understanding/applying/analyzing levels of Bloom's Taxonomy; results may depend on factors such as a heterogeneous student population, integration of high-quality instructional videos, a capable LLM (GPT-4), expert-crafted question-specific prompts, and content suited to this format; they "do not presume that structured AI tutoring will always outperform in-class active learning in all contexts, for example, those requiring complex synthesis of multiple concepts…
Preregistered crossover showing opposite comprehension effects for lower and higher baseline performers in a single session.
All four tools improved comprehension for lower baseline performers and reduced it for higher baseline performers; the Socratic chatbot helped lower performers most and the summary harmed higher performers most
Design
Preregistered within-subject randomized crossover with four GPT tools and one no-AI condition in one session
Sample
228 recruited; 195 analyzed
Population
US adults aged 18 to 22 recruited through Prolific
Measured
Comprehension accuracy on ACT-derived passages
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Single exposure to each condition; online convenience sample; subgroup definition and task-specific tools limit generalization
ChatGPT improved essay scores versus expert chat, writing tools, or none, but didn't boost knowledge gain, transfer, or motivation—possibly fostering reliance over deeper learning.
Fan and colleagues randomly assigned students to get help from ChatGPT, a human expert chat, writing analytics tools, or no extra tool while working on a writing task. Afterward, all four groups reported similar intrinsic motivation, but they differed significantly in how often and in what order they used self-regulated learning processes, the steps learners take to plan, monitor, and check their own work. The ChatGPT group improved their essay scores more than the other groups (F = 4.55, p = 0.005), but showed no significant advantage in knowledge gain or in transfer, applying that knowledge to new situations. The authors suggest ChatGPT's edge may be a short-term boost in task performance rather than deeper learning, and they raise concern that it might encourage technology dependence and "metacognitive laziness," reduced effort in monitoring one's own thinking.
Design
randomised experimental study, lab setting, 4 groups (ChatGPT, human expert, writing analytics tools, no extra tool)
Sample
117 university students total (CN group, 30 participants; AI group, 35 participants; HE group, 25 participants; CL group, 27 participants)
Population
university students; country/setting not reported in retrieved text; adult status not reported in retrieved text
Measured
Essay score improvement and knowledge/transfer test scores: behavioral (task performance/test performance, scored by researchers or via choice-question tests); Intrinsic Motivation Inventory (IMI - interest/enjoyment, perceived competence, effort/importance, pressure/tension): self-report; SRL process frequency and sequence (from learning trace data - navigational logs, click streams, mouse movement, keyboard…
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
not reported in retrieved text (authors instead propose implications for practice/policy, e.g. designing multi-task and cross-context studies in the future, but do not state study limitations in retrieved text)
ChatGPT access had no significant overall effect on learning, but substitutive use increased grades initially while decreasing them later, whereas complementary use improved understanding without harming breadth.
Researchers ran two pre-registered lab experiments with real incentives and found that giving students access to LLMs (AI chatbots like ChatGPT) made no significant difference to how much they learned overall. Looking closer at how students actually used the tool, those who used LLMs to replace their own study work covered more topics but understood each one less well, while those who used LLMs to add to their own study work understood topics better without covering more of them. LLMs also widened the gap between students: those who already knew more learned better with LLM help, while those who knew less learned worse.
Design
Field study (Study 1, two-way FE and FE2SLS panel analysis) + two pre-registered, incentivized, randomized between-subjects lab experiments (Studies 2 and 3), each with pre-test/learning-phase/post-test; exploratory pooled analyses combining Studies 2 and 3
Sample
Study 1: N=113 students (56 IS course, 57 BA course), unit of analysis student–question, 6,775 student–question observations total (6,594 used in main models). Study 2: 108 subjects recruited, 107 analyzed (54 vs 53 approx. per control/treatment, exact split…
Population
Study 1: graduate students in two programming courses (information systems and business analytics) at a public Dutch university, adults (graduate students), age not otherwise reported. Studies 2 and 3: subjects recruited from a laboratory participant pool at a public German university, described as…
Measured
Learning outcomes decomposed into: (1) overall learning outcome = Post-test score (number of correct answers out of 20 programming questions, administered before and after learning phase); (2) topic volume = number of correctly answered practice questions during the 45-minute Learning Phase (24 parts); (3) topic understanding = residual effect on Post-test after controlling for Learning Phase progress. Also:…
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
Authors state: exploratory findings "should ideally be replicated in controlled experiments and with additional field data." They "abstract away from the neurological building blocks of understanding, such as conceptual and factual understanding... and memory." Study 1 relies on a proxy measure (ChatGPT Similarity) since actual LLM use could not be directly observed ("Since we cannot directly observe whether a student used ChatGPT, ChatGPT Similarity is a proxy for this behavior"). Study 2 had an unintended methodological issue (copy-and-paste disabled), which the authors note affected LLM usage and necessitated…
GPT-4 tutoring tools improved practice-problem performance dramatically, but only the structured tutor prevented exam score drops—suggesting students misuse unguided AI as a shortcut without realizing it harms learning.
Bastani and colleagues randomly assigned students to practice with a GPT Tutor, a plain GPT Base chatbot, or no AI at all. During practice, GPT Tutor users scored 127% better and GPT Base users scored 48% better than the no-AI control group. But on the actual exam, GPT Base students did 17% worse than control, an effect the authors call "essentially eradicated" for GPT Tutor users, "though we still do not observe a positive effect." The authors suggest AI can boost performance but "substantially inhibit learning without appropriate guardrails" — students often used GPT Base as a "crutch," copying solutions instead of working problems, while GPT Tutor prompted "more substantive" engagement like asking for help; students didn't seem to realize the copying was hurting their learning.
"nearly 1,000 students" across "about fifty 9th, 10th, and 11th-grade classes"; main student-level regression uses 2,848 observations (student-level observations, not distinct students); problem-level specifications use 13,484 and 11,392 observations; 57…
Population
High school students (9th, 10th, and 11th grade) at a large high school in Turkey, Fall semester 2023–2024 academic year; minors (under 18)
Measured
Normalized student performance (grade) on assisted practice problems (Practice perf) and unassisted closed-book exam (Exam perf), graded by independent graders using teacher-designed rubric; also self-reported student perceptions of performance/learning via end-of-session survey; Herfindahl-Hirschman index (HHI) as measure of grade dispersion; NLP clustering of student-AI chat messages to categorize engagement…
Timing
after AI was removed
Found by
Both searches
Source basis
full text
Limit
The study focuses on two generative AI tutors for a single topic (mathematics) deployed in a single high school in Turkey; objective evaluation criteria for math problems are unavailable in other subjects such as writing; the deployment was carried out in Fall 2023 when generative AI was still very new, and modes of interaction may have changed as users become more familiar with these tools, and models have improved since then; the study focuses on short-term outcomes due to limitations imposed by the partner school, and studying long-term outcomes is a key direction for future research; while several analyses…
Dependent AI users reasoned worse on unaided tasks than never-users; autonomous users maintained baseline performance.
People who used AI by simply accepting its answers first, then reasoned worse on their own afterward than people who never used AI at all: their score dropped by 0.38 points on a 1-7 scale, a small effect (d = −0.28, p = 0.0002). People who used AI but had to generate their own answer first, before seeing AI's answer, did not show this drop. The researchers think this is about the order of workflow — whether you produce your own answer before or after seeing AI's — rather than about using AI itself; a "generation effect," where producing an answer yourself strengthens learning more than just reading one, fits this pattern but isn't confirmed as the actual mechanism. The result held up across several robustness checks, and people who reported a general tendency to offload thinking to others also tended to reason worse on their own.
Design
preregistered three-arm randomized experiment (self-generation-first, direct-adoption, no-AI), unaided transfer task 15–20 min later
Sample
900 randomized participants; primary analysis sample N = 659 (autonomous = 222, dependent = 225, no-AI = 212); per-protocol n = 585 (172/201/212... note text gives per-protocol …
Population
not reported in retrieved text (no country/setting/age specified beyond 'participants'; adult status not reported in retrieved text)
Measured
Behavioral: process log measures (outline submission, proportion of AI suggestions adopted vs. rewritten, number of AI calls, time on task) used for manipulation fidelity checks. Self-report: trait/process measures — ACO subscale (DACOS), Reflective Thinking Questionnaire, Critical Thinking Disposition Inventory, state metacognitive-monitoring scale, Intrinsic Motivation Inventory (interest/effort subscales), AI-use…
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
authors state the design cannot exclude a metacognitive pathway; the estimate belongs to the workflow contrast as a whole rather than to any single component (self-generation, elaboration, revision, effort, autonomy are confounded); a generation-effect account is consistent but cannot be weighed against alternatives; no mechanism is established; result is immediate near-transfer decrement, not demonstrated lasting de-skilling
Knowledge increased similarly across groups; reflective use and critical thinking were self-reported, and the nonrandomized study found no added knowledge gain from AI.
Researchers tracked 87 students across three classes—one using AI with guided tutoring, one using AI unguided, and one control group with no AI—and found all three groups gained knowledge equally, so adding AI gave no extra boost. Students who worked harder mentally in a useful way, called germane cognitive load, learned more, and that mental effort went up during the study. Students in the AI condition reflected more on their own thinking, and that reflection was linked to better critical thinking, though critical thinking itself didn't change overall. The authors conclude AI's value depends on how teachers frame its use and how well students regulate their own learning, and found no sign that AI widened gaps between weaker and stronger students.
Design
Quasi-experimental longitudinal study of three parallel classes
Sample
87 at baseline; 62 with T3 knowledge data
Population
Business informatics students
Measured
Knowledge test plus self-reported motivation load critical thinking and reflective use
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
Nonrandomized single-institution classes; small attrition-affected sample; several constructs are self-report and the motivation scale had low reliability
Students using ChatGPT3.5 for research experienced less mental strain but produced weaker reasoning and arguments in their recommendations, with no difference in how similar their justifications were.
Students who used ChatGPT3.5 to research a learning task felt less mental strain, called cognitive load, than students who used Google search, but their final recommendations showed weaker reasoning and argumentation. In the full paper, the ChatGPT group rated their mental effort lower than the Google group on a 1–7 scale — extraneous load 3.00 vs 3.96 (F = 8.26, p = 0.005) and intrinsic load 3.16 vs 4.43 (F = 16.48, p < 0.001). The authors suggest LLMs can ease mental effort but may not push students to engage deeply enough with content for high-quality learning. Recommendations and justifications were about equally varied, called homogeneity, between the two groups, suggesting LLMs didn't narrow students' range of perspectives.
Design
randomised between-subjects experiment, 2 arms (ChatGPT3.5 vs Google)
Sample
91 students (Web search n = 47; LLM n = 44); note: 92 recruited, 1 excluded for not following instructions
Population
university students; country/setting not reported in retrieved text; adults implied by 'university students' but not explicitly stated in retrieved text
Measured
Cognitive load (ICL, ECL, GCL): self-report (7-item Likert questionnaire, 1-7 agreement, Klepsch et al. 2017). Quality of justifications and recommendation direction: behavioral/performance (coded from participants' written text by two independent raters, k=0.92 and k=0.89 respectively). Prior domain knowledge: behavioral/performance (8-item multiple-choice knowledge test, adapted PKNT).
Timing
after AI was removed
Found by
Both searches
Source basis
full text
Limit
single task/topic (nanoparticles in sunscreen); small, imbalanced arms (Web search 47 / LLM 44)
Learning from LLM syntheses led people to exert less effort, report shallower understanding, and produce sparser, less trustworthy advice than learning from web search results.
In seven experiments with 10,426 participants, people who learned a topic from LLM summaries, instead of clicking through web links, said they built shallower knowledge and put in less effort learning. Those participants then gave advice that was sparser and more generic, and other people found that advice less informative and were less likely to actually use it. The authors suggest LLM syntheses can limit deeper, more original learning because they cut the need to dig up and combine information from original sources yourself. This pattern held even when the LLM summaries also included real-time web links, and it showed up across different topics and different LLM tools.
Design
Series of 7 RCTs (4 in main text: Experiments 1-4, plus 3 preregistered replications); between-subjects, random assignment to search platform/condition; Experiments 1, 2, 4 preregistered (per text); Experiment 3 not stated as preregistered
Sample
Total across 7 experiments N=10,426 (as stated in intro). Exp 1: 1,136 recruited, 1,104 analyzed after exclusions (Google vs ChatGPT arms). Exp 2: 2,016 recruited, 1,979 analyzed (Google vs ChatGPT arms). Exp 3: 253 recruited, 250 analyzed (Google-standard vs…
Population
Exp 1 & 2: Prolific panel members (online, adults - Exp1 mean age 42.40, SD 13.46; Exp2 mean age 41.28, SD 17.64); Exp 3: members of a behavioral lab panel at an east-coast US university, laboratory setting, adults (mean age 22.75, SD 8.65); Exp 4: MTurk Connect panel members, adults (mean age…
Measured
Self-reported depth of learning (Likert items: "I learned new things...", "comprehensive information...", "personal ownership over knowledge"); investment in advice formation (self-report items on thought/effort and ownership, plus time spent writing advice as behavioral proxy); time spent on search task (proxy for learning effort); number of queries submitted; advice content analyzed via NLP: word count (LIWC…
Timing
after AI was removed
Found by
Both searches
Source basis
full text (PMC12560091)
Limit
Authors note: findings arose in a specific experimental task context (learning to form advice to share with others) and different LLM applications may yield different results (e.g., no obvious advantage of web search over LLM for simple factual lookup); task contexts did not require deep prior knowledge, so effects "might not generalize... to learning in highly specialized domains—such as advanced science or math topics—where LLM syntheses might better help users comprehend and interpret complex topics compared with web links"; evidence centered on participants' self-reported lower depth of learning,…
Across 24 educational-chatbot studies, learning outcomes favored chatbots, with larger effects in higher education and shorter interventions; mixed ages, tools, and outcomes limit direct GenAI conclusions.
AI chatbots had a large effect on students' learning outcomes; the effect was greater for students in higher education compared to primary and secondary education; short interventions had a stronger effect on learning outcomes than long interventions, possibly due to novelty effects wearing off over time.
Design
meta-analysis of 24 randomized studies; examined moderating effects of educational level and intervention duration
Sample
k=24 studies; total participant N not reported in retrieved text
Population
students across primary, secondary, and higher education; setting/country not reported in retrieved text; mixed ages incl. minors
Measured
students' learning outcomes; specific instruments/tasks not reported in retrieved text
The ISAR framework proposes four ways AI can affect learning—inversion, substitution, augmentation, and redefinition—depending on instructional design, learner skill, and teacher practice. It proposes a model rather than testing one.
The paper argues that AI's benefits for learning are not inherent but depend on instructional implementation; it introduces the ISAR model distinguishing inversion (e.g., over-reliance leading to reduced cognitive engagement), substitution (equivalent instructional functionality), augmentation (additional cognitive learning support), and redefinition (fostering deep learning processes) effects of AI on learning, and emphasizes that successful AI integration depends on moderating factors such as students' AI literacy and educators' technological and pedagogical skills, calling for a systematic and evidence-based approach to AI in education.
Design
conceptual/theoretical reflection paper introducing the ISAR model; not an empirical study
This perspective argues that immediate AI-assisted performance is routinely mistaken for durable learning and calls for testing what remains after assistance is removed; it reports no new experiment.
The authors argue that generative AI can boost immediate task performance and reduce cognitive load, but these gains do not necessarily reflect durable learning; performance and learning are conceptually distinct, and current research on generative AI in education "pervasively conflates" the two. They note that reported performance improvements from AI assistance "diminished once the generative AI assistance was removed, indicating that immediate gains in performance might not reflect sustained improvements in student learning," and that heavy reliance on generative AI is associated with weaker argumentation/reasoning skills, "metacognitive laziness," lower learning autonomy, and increased technological dependence despite enhanced perceived confidence.
Design
Narrative/opinion (perspective) piece synthesising multiple prior studies and a meta-analysis; not itself an empirical study
Sample
not reported in retrieved text (this article reports no primary N; it cites a meta-analysis of "69 studies" [ref 1] and other individual studies without giving their Ns in this text)
Population
not reported in retrieved text (article discusses "tertiary education" / "K-12 and tertiary education" learners generally, citing other studies; no specific country, setting, or age breakdown given for a primary sample; cited meta-analysis covers K-12 and tertiary, implying mixed ages incl. minors…
Measured
Academic/task performance (varies by cited study); cognitive load; metacognition (e.g., "metacognitive laziness," offloading of evaluative tasks); intrinsic motivation/learning autonomy; self-efficacy and perceived confidence/technological dependence; quality of peer feedback; argumentation and reasoning skills — no single instrument specified in this text beyond descriptions (e.g., self-report of cognitive load,…
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
The authors state that research in this area suffers from "conceptual and methodological limitations," notably conflation of "performance" and "learning" constructs (e.g., the cited meta-analysis frames performance effects as "learning"), reliance on immediate/transient performance metrics rather than process-oriented assessments (retention/transfer/delayed recall tests), and a lack of longitudinal investigation of AI's long-term effects on knowledge retention, skill development, and interaction with prior knowledge.
Historical meta-analysis of 79 precollege reports; calculator integration did not produce the feared loss of paper-and-pencil skills.
Integrating calculators with mathematics instruction improved average students' paper-and-pencil basic skills and problem solving without the broad skill loss critics feared
Design
Meta-analysis
Sample
79 reports
Population
Precollege students
Measured
Paper-and-pencil basic skills problem solving and attitudes
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Studies were conducted in precollege education decades before generative AI and varied in calculator use and instructional design
Historical meta-analysis of 54 precollege studies; effects depended on whether calculators were integrated into instruction and assessment.
Skills improved when calculators were integral to instruction and testing; results without calculators in assessment were mixed but showed no general hindrance to mathematical skill development
Design
Meta-analysis
Sample
54 studies
Population
Precollege students
Measured
Operational computational conceptual and problem-solving skills plus attitudes
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Precollege evidence from heterogeneous calculator implementations; outcomes depend on whether the tool was integrated into instruction and assessment
Across the studies in this meta-analysis, generating information improved later memory by an average of 0.40 standard deviations compared with reading it.
Generating information produced a mean effect of 0.40 standard deviations over reading
Design
Meta-analysis
Sample
86 studies; 445 effect sizes
Population
Mixed populations
Measured
Memory after generating information versus reading it
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Mechanism evidence from varied generation tasks; it does not establish that every self-generation workflow or complex task benefits equally
Testing previously studied information improved later retention more than restudying it, with larger benefits from initial recall tests than recognition tests.
Testing previously studied information improved later retention; initial recall tests produced larger benefits than recognition tests and the pattern supports effortful processing
Design
Meta-analysis
Sample
Not reported in retrieved primary abstract
Population
Mixed populations
Measured
Retention after testing versus restudy
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Mechanisms remain contested; this is broad retrieval-practice evidence rather than a direct study of AI use
A 2018 meta-analysis reported near-zero recall and transfer effects; a later reproducibility critique left the transfer result intact while making the recall analysis less credible.
Perceptual disfluency had no effect on recall (d=-0.01) or transfer (d=0.03), lowered judgments of learning (d=-0.43) and increased learning time (d=0.52)
Design
Meta-analysis
Sample
25 articles; 3,135 participants
Population
Students
Measured
Recall transfer judgments of learning and learning time
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
A 2021 methodological commentary identified coding and effect-size errors; the direction of the null conclusion should be presented with that challenge visible
Exploratory hypermedia experiment illustrates that cognitive-load components are difficult to measure separately.
Multimedia presentation affected retention and attention, while the subjective measures could not reliably separate intrinsic extraneous and germane load across versions
Design
Two-phase hypermedia experiment with information-retention measures and eye tracking
Sample
92 in phase 1; 36 in phase 2
Population
First-year psychology students
Measured
Information retention subjective cognitive-load scales cognitive absorption and eye tracking
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Exploratory study of online-news layouts; the contested germane-load construct was difficult to measure and does not supply a simple rule that more effort improves learning
Theory review proposes matching task difficulty to material complexity and learner expertise; requires empirical validation.
The authors propose matching difficulty to material complexity and learner expertise: added difficulty may help low-interactivity tasks while guidance may help novices with complex material
Design
Theory review
Sample
Not applicable
Population
Not applicable
Measured
Review of difficulty cognitive load and long-term retention
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
The proposed integrated model is theoretical and requires controlled empirical validation
Questions still unanswered
Do the immediate gains from designed AI tutors survive after days, weeks, or months?
Which results come from the model, the instructional design, additional feedback, time on task, or the learner's interaction choices?
Can adaptive systems reliably identify who will benefit before exposing higher-performing learners to tools that may reduce comprehension?
Do self-generation, retrieval practice and explanation-seeking cause the better unaided outcomes seen in some AI workflows?
What happens with repeated ordinary use outside a controlled lesson?
Do the patterns hold for working adults learning on the job, experienced professionals, older adults, and people with cognitive or communication impairments?
Does AI change transfer to new domains rather than performance on closely related tests?
How do effects change as models and interfaces improve?
AI can help or hinder the development of a new skill depending on how it is used. One observational field study raises a credible warning about established professional performance, while surveys and theory identify concerns that have not yet been demonstrated behaviorally.
Across three workplace experiments, Copilot access increased completed tasks by 26.08% on average, with larger gains for less-experienced developers; the studies did not test skill retention.
Copilot access increased completed tasks by 26.08 percent on average with larger gains among less-experienced developers
Design
Three field randomized controlled trials
Sample
4867
Population
Professional software developers at Microsoft Accenture and one Fortune 100 company
Measured
Completed work items under GitHub Copilot access
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
The experiments measured productivity while the tool was available; they did not test unaided skill acquisition retention or decay
Among 69 learners ages 10–17, Codex improved assisted code-writing and did not reduce average unaided performance one day or one week later, although single-prompt generation produced the weakest code-modification results.
Codex improved assisted code-authoring performance and did not reduce average unaided post-test performance; the single-prompt generation pattern produced the lowest code-modification correctness
Design
Controlled experiment; Codex access or no code generator across 10 sessions over three weeks; unaided post-tests one day and one week later
Sample
69
Population
Learners aged 10 to 17 with no prior text-based programming experience
Measured
Code-authoring performance during training and unaided code-modification and code-comprehension post-tests
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Small sample of minors learning introductory Python; subgroup benefits depended on prior Scratch knowledge; the study concerns new skill acquisition rather than established adult skill
Across five preregistered studies, AI-supported writing practice improved unaided writing one day later more than unaided practice; studying an AI example performed about as well as practicing with AI.
AI-supported practice improved later unaided writing more than unaided practice; merely studying an AI-generated example performed similarly to practicing with AI
Design
Five preregistered studies including randomized writing-practice experiments
Sample
Study 2: 2238; Study 4: 2997; Study 5: 2003
Population
Adults recruited through Prolific
Measured
Unaided cover-letter writing after practice including a one-day delayed test
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Preprint using one writing genre and short follow-up; it studies adult skill acquisition rather than loss of established expertise
In a randomized study of 52 junior developers, the AI group scored 50% on an unaided quiz versus 67% after hand coding. Within the AI group, more engaged interaction patterns were associated with better learning than full delegation; that pattern comparison was observational.
The AI group scored 50 percent on the unaided quiz versus 67 percent for hand coding; full delegation provided some productivity benefit with weaker learning while cognitively engaged interaction patterns preserved learning
Design
Randomized experiment
Sample
52
Population
Junior software developers learning an unfamiliar asynchronous programming library
Measured
Conceptual understanding code reading debugging and task completion
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Small preprint experiment on one unfamiliar programming library; interaction-pattern findings are partly observational within the AI group
Cheaper AI assistance increased reliance on it; participants who used it more performed worse once removed, suggesting skipped independent problem-solving weakens skill development.
When AI help cost less, participants asked for it more often. Those who leaned on AI during the assisted phase then did worse on their own once the AI was taken away, and their later unassisted performance was overestimated when guessed from how well they'd done with AI help. Using a model that separated starting skill, later skill, and skill change, the authors suggest that people who kept solving problems on their own during the AI phase gained more real ability, meaning skill-building seems weaker when AI substitutes for independent reasoning. In the full paper, the 75 participants who kept solving problems without AI during the middle phase improved their rate from 2.03 to 3.86 correct items per minute — a 90.2% gain (t = 7.40, p < 0.01).
Design
Controlled experiment with pre-assistance assistance and immediate post-assistance phases; AI request cost was randomized
Sample
124
Population
US adults recruited through Prolific with at least an undergraduate degree
Measured
Logic-puzzle accuracy response time reward rate logged AI requests and independent reasoning time
Timing
after AI was removed
Found by
Claude Science
Source basis
full text
Limit
Single-session simulated perfectly accurate AI; important AI-use comparisons were observational; the low-cost versus no-AI change test used a one-sided p<.10 threshold; DOI and arXiv records leave publication status to verify
Across 1,443 standard colonoscopies, adenoma detection fell from 28.4% before workplace AI adoption to 22.4% afterward. With no concurrent control group, the study cannot establish that AI exposure caused the decline.
Adenoma detection during standard colonoscopy fell from 28.4% before AI exposure to 22.4% afterward, a six-percentage-point decline. The observational study had no concurrent control group. A later correction added colonoscopy indication to a secondary regression across all procedures, changed two secondary estimates, and left the journal's interpretation unchanged.
Design
Retrospective multicentre observational before-and-after comparison of non-AI colonoscopies before and after workplace AI adoption
Sample
1443 colonoscopies: 795 before and 648 after; 19 endoscopists
Population
Experienced endoscopists at four centres in Poland
Measured
Adenoma detection rate during standard colonoscopy without AI
Timing
after AI was removed
Found by
Both searches
Source basis
full text (author accepted manuscript) and corrected publisher appendix
Limit
No concurrent control group and no random assignment; colonoscopies rather than people are the observations; ADR is a performance outcome and cannot identify the cognitive mechanism. The 2025 correction added indication to the supplementary all-colonoscopy model and changed the AI-assisted and age estimates; the journal states that the interpretation is unchanged.
Pulmonologists recognize AI's clinical potential (87%) but worry substantially about procedural deskilling (73%), broadly endorsing mitigation strategies and rating deskilling a top research priority (89%).
Marchi and colleagues surveyed 118 interventional pulmonologists from 10 countries and found 87% saw real clinical potential in AI, but 73% worried AI could erode their procedural skills over time, a concern called "deskilling." Deskilling ranked as the top priority for future research, with 89% agreement, and respondents broadly backed ways to guard against it: training without AI assistance (84%), simulation-based practice (86%), long-term monitoring of skills (78%), and formal governance rules (70%). Only 38% had previously heard of "automation bias," blindly trusting a machine's output, but once the term was explained, 81% said it resonated, a 43-point jump. The authors stress this survey captures opinions and attitudes only, not measured skill loss, and they explicitly warn it should not be taken as proof that AI actually causes deskilling in this field; no statistical significance testing was done.
Design
International cross-sectional questionnaire survey
Sample
118
Population
Interventional pulmonologists in 10 countries
Measured
Unvalidated 12-item survey of perceived deskilling risk and mitigation preferences
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Convenience sample with 11 or 12 respondents per country; measures perceptions rather than observed skill; hypothetical framing and an unvalidated instrument limit inference
Reported AI dependence was associated with perceived skill atrophy; raw use intensity alone was not, and the cross-sectional survey cannot establish direction or cause.
In a cross-sectional survey of 143 knowledge workers, reported AI dependence was associated with greater perceived skill atrophy (β = 0.488, 95% CI 0.338 to 0.635). Raw AI-use intensity was not significantly associated with perceived atrophy, so the study does not show that heavier use caused skills to erode over time.
Design
Two-wave survey
Sample
143
Population
Knowledge workers across five countries
Measured
Self-reported AI use dependence and two-item perceived skill-atrophy scale
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Self-report and correlational model; perceived erosion is not a behavioral measure of skill change
This theory paper proposes that AI can weaken established skills or impede new skill development when it replaces practice; it reports no original data.
The paper proposes that AI could accelerate decay of established skill and hinder development of new skill when it replaces practice; it reports no original study
Design
Theoretical perspective
Sample
Not applicable
Population
Not applicable
Measured
Review of skill decay deliberate practice automation and AI assistance
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Hypotheses derived largely from earlier automation and skill-learning research; direct AI evidence was sparse when it was published
This perspective argues that full offloading can impede skill acquisition and contribute to decay, while the risk depends on how AI is used; it reports no original experiment.
The authors argue that full offloading can impede skill acquisition and contribute to decay while risks depend on how AI is used; they distinguish domain-specific skills from basic cognitive abilities
Design
Three-page narrative perspective
Sample
Not applicable
Population
Not applicable
Measured
Narrative synthesis of skill acquisition skill decay cognitive offloading and basic cognitive abilities
Timing
not applicable
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
No original data and no systematic-review method; conclusions depend on a small selected literature; long-term effects on basic abilities remain unknown
Among 66 pilots, recent manual-flying practice predicted performance more strongly than total or long-term automated-aircraft experience, with airspeed tracking especially vulnerable.
Recent manual-handling experience predicted performance more strongly than total or long-term automated-aircraft experience; airspeed tracking was especially vulnerable
Design
Multi-study PhD thesis; main manual-flying performance study plus a 9-pilot cognitive task analysis and measure validation
Sample
66 in the main performance study; thesis also includes a 9-pilot cognitive task analysis
Population
32 first officers; 27 captains; 7 training captains in the main performance study
Measured
Manual-flying performance including airspeed tracking plus recent and cumulative flying experience
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Thesis contains several component studies; the prior source record incorrectly paired one component's N=9 with the main performance conclusion
After prolonged automation, pilots largely retained practiced hand-eye control while navigation, failure recognition, and diagnosis were more vulnerable.
Well-practiced hand-eye and manual-control skills were largely retained while cognitive navigation failure recognition and diagnosis were more vulnerable
Design
Level-D simulator study across three automation conditions
Sample
16
Population
Active Boeing 747-400 pilots
Measured
Manual control instrument scanning navigation failure recognition and diagnosis
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Small expert sample in one aircraft type; historical automation comparison rather than generative-AI evidence
Recent manual-flight practice predicted pilots' fine-motor performance more strongly than time since flight school, cumulative experience, or experience on a particular aircraft type.
Recent flight practice was a stronger predictor of fine-motor performance than time since flight school or cumulative and type-specific experience
Design
Cross-sectional observational study
Sample
126
Population
Airline pilots from different fleets
Measured
Fine-motor manual-flying performance and recency of practice
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Cross-sectional association cannot isolate automation as the cause; historical aviation setting limits transfer to knowledge work
Questions still unanswered
What happens to established expertise after ordinary generative-AI use for months or years?
Which components of a professional skill change: factual knowledge, pattern recognition, procedural fluency, diagnosis, error detection, or the ability to recover when automation fails?
If performance declines after sustained reliance, does it recover when people resume unaided practice?
Is there a dose-response relationship between frequency or depth of AI reliance and later unaided performance?
Which interaction patterns preserve practice while still providing useful assistance?
Do people accurately perceive changes in their own skill, or do self-reports diverge from behavioral tests?
Does AI displace existing skills while creating new ones in verification, delegation, synthesis, or supervision?
How do baseline expertise, age, task complexity, model reliability, and organizational safeguards change the result?
Can longitudinal workplace studies separate AI exposure from time trends, hiring, task mix, and training changes?
AI advice often improves performance compared with working alone when the advice is correct and the task fits the model. Human-AI teams rarely outperform the stronger party on average, and incorrect or overly trusted advice can sharply reduce judgment.
Preregistered experiment found that advice shifted trolley-dilemma judgments similarly whether attributed to ChatGPT or a human moral advisor.
Advice shifted moral judgments; influence did not differ significantly by whether advice had a justification or was attributed to ChatGPT versus a human advisor
Design
Large preregistered online vignette experiment
Sample
1269
Population
US adults
Measured
Change in trolley-dilemma judgment after reasoned or unreasoned advice attributed to ChatGPT or a human moral advisor
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
One stylized moral dilemma; advice acceptance in a vignette does not establish durable value change
Cognitive forcing functions reduced overreliance on incorrect AI predictions versus simple explanations, but didn't improve overall team performance and may have favored high-cognition participants.
Researchers tested designs that force people to pause and think before accepting an AI's suggestion, comparing them to simpler AI explanation approaches and to no AI at all. The forcing designs cut overreliance — trusting the AI's answer without enough scrutiny — more than the simple explanations did, though exact numbers weren't reported. But participants rated the forcing designs least favorably, exactly the ones that worked best. The forcing designs also helped more for people higher in Need for Cognition, a trait reflecting how much someone enjoys effortful thinking, suggesting a person's cognitive motivation shapes how well these AI designs work.
Design
Between-subjects experiment comparing cognitive-forcing explainable-AI and no-AI designs
Sample
199
Population
US Mechanical Turk adults
Measured
Decision accuracy overreliance preference trust mental demand and complexity
Timing
with AI present
Found by
Claude Science
Source basis
full text (arXiv author manuscript)
Limit
Single low-stakes nutrition task with simulated 75-percent-accurate AI; benefits differed by Need for Cognition
Harder tasks, easier-to-check explanations, and larger accuracy incentives reduced overreliance under the tested conditions.
Across five studies with 731 people doing a maze task with a simulated AI, participants relied too much on the AI's advice more often when checking its explanation was harder or the task itself was harder, and relied on it less when they were paid money for getting things right. A fifth study used an adapted effort-based test, one that measures how much mental effort someone will spend for a given payoff, and found the same pattern. The authors suggest this means overreliance isn't just an automatic mental bias — people seem to be weighing the effort of checking an explanation against its benefits and deciding strategically whether to bother. Exact statistical values (like p-values or effect sizes) weren't given in the abstract.
Design
Five experiments manipulating task difficulty explanation difficulty and incentives
Sample
731
Population
Online adults
Measured
Acceptance of incorrect AI predictions and cognitive-effort tradeoffs
Timing
with AI present
Found by
Claude Science
Source basis
full text of the arXiv preprint version (arXiv:2212.06823) of the same work
Limit
Controlled low-stakes maze task with simulated AI; headline numeric effects were not isolated in the retrieved text
Static LLM analysis did not help; adaptive selection improved accuracy and reduced overreliance in controlled tasks.
Showing all or sequential LLM analyses did not improve performance; adaptively selecting fewer analyses improved accuracy and reduced overreliance relative to several baselines
Design
Two randomized human-subject experiments across income and recidivism tasks
Sample
284 in the first experiment; 844 in the evaluation experiment
Population
US adults recruited through Prolific
Measured
Decision accuracy overreliance and underreliance
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Controlled prediction tasks; the adaptive method models an average decision maker and can also be used to steer decisions toward a chosen target
Three studies show the value of explanations depends on the task and that error recovery remains incomplete.
Explanations raised confidence without added accuracy in RAVEN; in LSAT tasks LLM explanations produced the highest accuracy and error recovery but recovery remained below 50 percent
Design
Three controlled studies: one multi-stage and two between-subjects
Sample
27 + 100 + 80
Population
University STEM students
Measured
Accuracy confidence reliance and recovery after incorrect AI advice
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Preprint; small student samples; results changed across visual reasoning and verbal reasoning tasks
Human–AI teams outperformed humans alone but typically underperformed whichever partner was stronger; gains depended on task type.
Researchers combined 106 experiments where people did tasks alone, AI did them alone, or people and AI worked together. On average, teaming up beat working alone, with a medium-to-large g = 0.64 (p = 0.000, 95% CI 0.53 to 0.74) — this is called human augmentation. But teams did not beat whichever was better, human-alone or AI-alone, showing a small negative effect, g = −0.23 (p = 0.005, 95% CI −0.39 to −0.07) — meaning no true "synergy" on average. Whether a task was a decision task or a creation task, and whether the human or the AI was individually better, changed how well the team did; giving the AI's confidence level or an explanation for its answer did not make a difference.
Design
Preregistered systematic review and meta-analysis
Sample
106 experiments; 370 effect sizes
Population
Human participants across included experiments
Measured
Performance of humans alone AI alone and human-AI combinations
Timing
with AI present or not reported
Found by
Both searches
Source basis
full text (PMC11659167)
Limit
Very high heterogeneity; publication bias possible; included studies varied substantially in task and design
A mathematical ensemble of existing physician and LLM answers outperformed human-only and LLM-only ensembles because their errors differed; this was not live collaboration.
Researchers mathematically combined diagnoses already supplied by physicians and LLMs. Weighted hybrid ensembles were more accurate than human-only or LLM-only ensembles because the two groups missed different cases. The study did not test physicians interacting with AI in real time.
Design
Retrospective analysis of weighted human LLM and hybrid ensembles with repeated cross-validation
Sample
2,133 cases; 3,669 physicians; hybrid analyses used 1,928 cases
Population
Physicians medical trainees and medical students on Human Dx
Measured
Top-1 top-3 and top-5 diagnostic accuracy plus mean reciprocal rank
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Mathematical aggregation of existing diagnoses rather than live collaboration; curated text vignettes; authors describe gains as a likely upper bound for practice
Physicians using ChatGPT alongside standard tools didn't improve their diagnostic reasoning compared to standard tools alone, though the AI itself performed better than either group of doctors.
Physicians given GPT-4 scored similarly to the conventional-resources group. The adjusted time estimate was 82 seconds faster, but the interval ranged from 195 seconds faster to 31 seconds slower, so the study did not establish a speed benefit. GPT-4 alone scored 16 percentage points above the conventional-resources group.
Design
Randomized single-blind multi-site trial of GPT-4 access versus conventional resources
Sample
50
Population
US-licensed physicians
Measured
Structured diagnostic-reasoning score time and diagnosis accuracy
Timing
with AI present
Found by
Claude Science
Source basis
full text (PMC11519755)
Limit
Six curated vignettes; small sample; unstandardized tool use; LLM-alone comparison was not a randomized human arm
Physicians using GPT-4 outperformed those relying on conventional clinical resources on management reasoning tasks, with the benefit persisting even after accounting for extra time spent.
Physicians using GPT-4 scored 43.0% versus 35.7% with conventional resources. The adjusted between-group estimate was 6.5 percentage points (95% CI 2.7 to 10.2); the raw mean difference was 7.3 points. GPT-4 alone and GPT-4-assisted physicians scored similarly.
Design
Prospective randomized rater-blind trial of GPT-4 access versus conventional resources
Sample
92
Population
Practicing and resident physicians
Measured
Expert-scored management reasoning on clinical vignettes
Timing
with AI present
Found by
Claude Science
Source basis
full text (PMC12380382)
Limit
Vignette task; rubric not validated outside the study; only correct content earned points while incorrect content was not penalized; a 2025 author correction concerns equal-contribution footnotes and does not change results
Randomized public self-assessment study found strong model-alone performance but no benefit when people used the models.
The LLMs alone named a relevant condition in 94.9 percent of cases but people given access named fewer than 34.5 percent; control participants had 1.76 times the odds of identifying a relevant condition
Design
Preregistered randomized study of three LLM assistants versus control
Sample
1298
Population
UK adults
Measured
Identification of relevant conditions and appropriate level of care
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Self-assessment through a chat interface differs from clinical care; failures included incomplete information model misinterpretation and users' difficulty selecting the best suggestion; a 2026 author correction fixes a figure-axis label; the corrected figure version should be used
Radiologists distrusted AI advice but followed it as readily as human advice; both groups' accuracy depended on advice correctness, not source.
Radiologists in this study rated advice as lower quality when they were told it came from AI rather than from a human, even though the advice was equally accurate either way — a bias the authors call algorithmic aversion. Non-expert participants showed no such bias, rating AI and human advice similarly. In both groups, how good they judged the advice to be tracked how confident they felt in their diagnosis (experts: r(1102) = 0.43, p < 0.001; non-experts: r(1014), p < 0.001, exact correlation not stated). Despite radiologists' distrust of AI-labeled advice, their actual diagnostic accuracy wasn't hurt by source — accurate advice boosted performance by 40.10% for experts and 37.53% for non-experts, with no difference based on whether the advice was labeled AI or human.
Design
Preregistered web experiment with advice source between participants and advice accuracy within participants
Sample
265
Population
Practicing radiology and internal or emergency medicine physicians
Measured
Diagnostic accuracy advice-quality ratings and confidence
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Eight web-based chest-X-ray cases; no no-advice control; simulated advice source; dynamic clinical work was not reproduced
Dell'Acqua 2022Laboratory for Innovation Science at Harvard working paper
AI studyPrimary studypreprint
Higher-quality AI reduced recruiters' effort and produced worse human assessments than lower-quality AI.
Recruiters given higher-quality AI exerted less effort and made less accurate assessments than recruiters given lower-quality AI; experienced recruiters drove the effect
Design
Preregistered randomized field experiment with four AI-quality conditions
Sample
181
Population
Professional HR recruiters
Measured
Accuracy effort and selection across 44 resume reviews per recruiter
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Working paper; hiring simulation for one role; AI quality was disclosed to participants
Radiologists became more accurate when the AI gave correct suggestions, but also more likely to follow incorrect AI advice—especially inexperienced radiologists—showing automation bias from relying on AI-based systems.
When radiologists rated mammograms and got a correct AI-suggested BI-RADS category (a breast cancer risk score), their accuracy was high: 79.7% for inexperienced readers, 81.3% for moderately experienced ones, and 82.3% for very experienced ones. When the AI suggestion was wrong instead, accuracy dropped sharply for everyone, down to 19.8%, 24.8%, and 45.5% for those same groups (all differences significant, P < .001 or P = .003). The authors say this suggests radiologists are prone to automation bias, meaning they lean on AI advice even when it's wrong. Inexperienced radiologists followed incorrect higher-risk AI suggestions more strongly (bias score 4.0 ± 1.8) than moderately experienced (2.4 ± 1.5, P = .044) or very experienced radiologists (1.2 ± 0.8, P = .009), suggesting more experience might lessen this bias.
Design
Prospective experiment with randomized mammogram sets and deliberately correct or incorrect purported-AI suggestions
Sample
27 radiologists reading 50 mammograms
Population
Radiologists at three experience levels
Measured
BI-RADS accuracy bias reading time perceived AI accuracy and confidence
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Small reader sample; purported rather than deployed AI; controlled image-reading task
Standard AI diagnostic support improved clinician accuracy, especially with explanations included; however, biased AI predictions worsened accuracy even when explanations were provided.
Clinicians reading medical case vignettes got the right diagnosis 73.0% of the time on their own (95% CI, 68.3% to 77.8%). When a standard, unbiased AI gave them a prediction, accuracy rose by 2.9 percentage points (95% CI, 0.5 to 5.2), and rose by 4.4 points (95% CI, 2.0 to 6.9) when the AI also showed an explanation, a visual highlight of what it used to make its call. But when the AI was systematically biased, meaning it made a consistent pattern of errors, accuracy fell by 11.3 points (95% CI, 7.2 to 15.5), and still fell by 9.1 points (95% CI, 4.9 to 13.2) even when it showed its explanation — only a 2.3-point improvement (95% CI, -2.7 to 7.2) that was not statistically significant. The authors conclude that while helpful AI improves diagnosis, biased AI harms it, and these common image-based explanations did not fix that harm.
Design
Randomized clinical-vignette survey comparing AI predictions with or without image explanations
Sample
457
Population
US hospitalist physicians nurse practitioners and physician assistants
Measured
Diagnostic accuracy on acute respiratory-failure vignettes
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Vignette study; explanations were one image-based method; results do not establish effects in live care
Incentivized experiment found overreliance on advice labelled as AI even when it conflicted with contextual information.
Participants overrelied on advice labelled as AI even when it conflicted with contextual information and their own assessment leading to inefficient outcomes
Design
Incentivized interactive behavioral experiment
Sample
Not reported in retrieved primary abstract
Population
Not reported in retrieved primary abstract
Measured
Reliance on AI-labelled advice and decision efficiency
Timing
with AI present or not reported
Found by
Claude Science
Source basis
abstract only
Limit
Abstract-only extraction; sample task and arm details were not reported in the retrieved text
AI improved performance on tasks within its capabilities—more completed, faster, higher quality—but reduced success on complex tasks beyond its frontier, suggesting real limits to AI support.
Across 18 tasks inside GPT-4's capability frontier, participants completed 12.2% more tasks, worked 25.1% faster, and produced higher-quality solutions. On a task outside the frontier, AI users were 19 percentage points less likely to reach the correct answer than people working without AI.
Design
Preregistered field experiment
Sample
758
Population
BCG consultants
Measured
Completion speed task quantity quality and correctness
Timing
with AI present
Found by
Both searches
Source basis
abstract only
Limit
One model era and consulting setting; researchers selected one task designed to sit outside the model's capability frontier
People using a fallible spell-checker accepted incorrect advice and left more errors in the text; the experiment measured error detection during use rather than retained proofreading skill.
Participants trusted incorrect spell-check advice and left more errors when assistance was active; higher-verbal-ability users were especially affected
Design
Between-subjects experiment with spell-check available or unavailable
Sample
65
Population
33 undergraduate and 32 graduate students
Measured
Errors remaining after proofreading text with a fallible spell checker
Timing
Not reported in the retrieved record
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Measures reliance and error detection during assisted performance; no withdrawal phase or later skill test
Questions still unanswered
Which forms of expertise help people detect incorrect AI advice, and under what task conditions?
Can people learn when a model is outside its capability frontier without repeated costly errors?
Which interface interventions improve accuracy in real work while remaining usable enough to be adopted?
How does reliance change after months of use, as people learn the tool and the tool changes?
Does reduced checking effort persist when users know that AI quality varies across tasks?
How do accountability, time pressure, incentives, and organizational review change reliance?
When human and model errors are complementary, which aggregation methods preserve that benefit in a live workflow?
Do people recognize when advice has shifted their judgment, including moral and high-stakes judgments?
Which results transfer from vignettes, mazes, and simulations to decisions with real consequences?
How do current generative systems compare directly with earlier automated decision aids under matched tasks?
People can detect some differences between correct and incorrect AI answers, but their confidence is easily distorted by explanations and perceived fluency. In the clearest field result, experienced developers believed AI had sped them up even though it measurably slowed them down.
Gagan Bansal et al. 2019Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP)
AI studyPrimary studypeer-reviewed
People formed more accurate mental models and achieved better human-AI team performance when the simulated AI had a simpler, more predictable error boundary.
Simpler and more predictable error boundaries produced better mental models and higher team performance; performance declined as human-visible task features increased
Design
Series of controlled experiments manipulating a simulated AI's error boundary
Sample
25 Mechanical Turk workers per condition; total not isolated in the retrieved text
Population
Amazon Mechanical Turk workers
Measured
Mental model of the AI's errors and human-AI team accuracy
Timing
Not reported in the retrieved record
Found by
Claude Science
Source basis
full text (publisher PDF via Unpaywall)
Limit
Pre-LLM simulated AI; 25 workers per condition; total sample and headline effect sizes not isolated in the retrieved text
Default explanations made people overestimate LLM accuracy, and longer explanations raised confidence without raising accuracy; confidence-calibrated explanations narrowed the gap.
Default explanations led people to overestimate LLM accuracy and longer explanations raised confidence without raising accuracy; confidence-calibrated explanations narrowed calibration and discrimination gaps
Design
Three behavioral experiments
Sample
301 across experiments 1 and 2
Population
US adults recruited through Prolific
Measured
Accuracy estimates expected calibration error and discrimination between correct and incorrect LLM answers
Timing
Not reported in the retrieved record
Found by
Claude Science
Source basis
full text (publisher PDF via Unpaywall)
Limit
Single-session multiple-choice and short-answer tasks; improvement came from changing the model output rather than training users
AI users performed better but overestimated their scores; self-reported AI literacy did not improve calibration, and the task measured LSAT reasoning rather than learning.
Across two LSAT-reasoning studies, AI users performed better while overestimating their scores. Confidence and AI-literacy measures were self-reported; higher self-reported AI literacy did not improve calibration. The studies measured reasoning performance with AI, not learning or later unaided performance.
Design
Two studies including an AI-versus-no-AI comparison with incentivized self-assessment
Sample
Study 1: 246; Study 2: 452
Population
US adults recruited through Prolific
Measured
LSAT accuracy estimated score trial-level confidence calibration and AI literacy
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Preprint; one LSAT task domain; Study 1 uses a historical no-AI comparison and Study 2 assignment procedure is unclear in the retrieved text
Allowing AI coding tools increased task completion time by 19%, contrary to developers' expectations; the slowdown held across robustness checks but may reflect the study's setting of experienced developers.
Developers randomly assigned to use AI tools on coding tasks took 19% longer to finish than those who worked without them. Before starting, the developers had guessed AI would make them 24% faster, and even afterward they still believed it had made them 20% faster. Outside experts were even more optimistic: economics experts predicted AI would cut task time by 39%, and ML experts predicted a 38% cut. The researchers call this slowdown surprising, say it's unlikely to be mainly a quirk of the experiment's design, and found evidence for 5 of 21 possible contributing factors, with mixed or unclear evidence for 10 more.
Design
Randomized issue-level field experiment
Sample
16 experienced developers completing 246 issues
Population
Experienced open-source developers
Measured
Task completion time plus pre-task forecasts and post-hoc estimates
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Small participant sample in mature repositories; completion time partly self-reported; setting-specific preprint
Among 98 university students, ChatGPT-assisted writers incorporated more correct information than independent writers but no less incorrect AI-generated information. Across conditions, smaller confidence-performance gaps were associated with more correct expert information, while larger gaps and stronger beliefs in ChatGPT as an authority were associated with more incorrect AI claims.
Students who wrote with ChatGPT-4o incorporated more correct information than independent writers from both expert-written (d=0.64) and ChatGPT-generated articles (d=0.95), but incorporated no less incorrect AI-generated information. Across both conditions, smaller confidence-performance gaps were associated with more correct expert information (R-squared=12%); larger gaps (R-squared=10%) and stronger epistemic beliefs about ChatGPT (R-squared=12%) were associated with more incorrect AI claims.
Design
Two-condition experiment: independent versus ChatGPT-assisted academic writing
Sample
98 university students
Population
University students
Measured
Correct and incorrect information integrated into writing plus metacognitive-bias and epistemic-belief measures
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
Small university-student sample; no delayed outcome; metacognitive bias combines confidence with measured performance; the regression associations across conditions do not establish that metacognitive bias or epistemic beliefs caused information uptake.
Heavier reported AI use was associated with more self-reported offloading and lower self-reported critical thinking; the cross-sectional study cannot establish causality.
Heavier reported AI use was associated with more reported offloading and lower reported critical thinking; direction and causality cannot be established
Design
Cross-sectional mixed-method survey and interviews
Sample
666 survey respondents; 50 interviews
Population
Participants across age and education groups
Measured
Self-reported AI use cognitive offloading critical thinking and deep-thinking activity
Timing
with AI present
Found by
Both searches
Source basis
full text
Limit
All key outcomes are self-report; cross-sectional; the paper reports inconsistent correlations for the same relationship; a 2025 correction replaced a duplicated Table 4
Higher confidence in GenAI predicts less critical thinking; higher self-confidence predicts more. GenAI shifts critical thinking toward verification and integration.
People who trusted the AI tool more tended to think critically about its answers less, while people who trusted their own skills more tended to think critically more, according to Lee et al.'s survey of GenAI-assisted tasks. The authors suggest this shift means critical thinking now looks less like solving the problem yourself and more like checking the AI's information, blending its answers with your own, and overseeing the task overall. No exact statistical values, like coefficients, confidence intervals, or p-values, were reported.
Design
Survey with quantitative models and qualitative analysis of work examples
Sample
319 knowledge workers; 936 analyzed task examples
Population
Knowledge workers
Measured
Self-reported critical-thinking enaction effort confidence reflection and trust
Timing
with AI present
Found by
Both searches
Source basis
full text
Limit
Self-report survey without behavioral performance or a control group; examples are clustered within participants
Generative AI use linked two distinct offloading patterns: dependent offloading with weaker motivation and agency loss; autonomous offloading maintained motivation but showed no agency transfer.
Zhu and colleagues surveyed people three times over a period to see how relying on AI for thinking tasks relates to how people feel about their own thinking. They found two different styles of "cognitive offloading" (letting AI do mental work for you): a dependent style and an autonomous style, and these related to different outcomes down the line. Two separate pathways seemed to explain this: how much people felt they'd handed over control of their own thinking, and how much they were internally motivated versus just relying on the tool; having stronger metacognitive awareness, meaning noticing and understanding your own thinking, might soften some of the downsides tied to the dependent style. Both styles felt equally helpful in the moment, which suggests that engagement with AI that may be unhelpful in the long run could be hard for users to spot from how it feels right away. No specific numbers, confidence intervals, or effect sizes were reported in the text provided.
Design
Three-wave time-lagged survey at two-week intervals
Sample
589
Population
Primarily Chinese university students with 73 early-career workers
Measured
Self-reported offloading monitoring agency motivation capability creativity processing and judgment
Timing
after AI was removed
Found by
Both searches
Source basis
full text
Limit
All constructs and outcomes were self-reported; correlational design without baseline outcomes; new scales and heterogeneous tasks
Questions still unanswered
Do people learn to recognize a model's failure patterns after repeated real-world use?
Does calibration transfer when the model, interface, or task changes?
Can users accurately estimate how AI affects speed, quality, learning, and originality at the same time?
Which forms of feedback correct mistaken beliefs about AI's benefit without discouraging useful adoption?
Does self-reported AI literacy correspond to demonstrated calibration skill?
Can metacognitive training produce durable improvements after the intervention ends?
How do organizational incentives and time pressure affect whether people notice and report AI-driven errors?
When people say they used critical thinking, which observable behaviors distinguish verification from post-hoc acceptance?
Can process traces, confidence judgments, and performance measures be combined without treating any one as a complete account of cognition?
AI-generated messages can shift reported beliefs in controlled experiments, including during writing tasks that are not framed as persuasion. A few studies find effects weeks or months later, while most measure attitudes immediately after one exposure and say little about behavior outside the experiment.
Biased autocomplete shifted attitudes during ordinary writing tasks and warnings did not remove the effect.
Biased autocomplete shifted attitudes toward the assistant's position; warnings before or after exposure did not reduce the effect and most participants did not recognize the bias or influence
Design
Two preregistered randomized experiments using biased AI autocomplete
Sample
2,582 across two experiments: 1,485 and 1,097
Population
Adults recruited through Prolific
Measured
Post-writing attitudes and recognition of bias and influence
Timing
immediately after the writing task
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Immediate self-reported attitudes after writing; selected societal issues and one autocomplete interface; no delayed or behavioral outcome
Evidence-based dialogue reduced confidence in conspiracy and other unwarranted beliefs regardless of whether the messenger was framed as AI or human.
Dialogue reduced conspiracy-belief certainty by about 10 points and other unwarranted-belief certainty by about 5 points with no significant differences by messenger framing or tone
Design
Preregistered randomized 2x2 experiment varying AI-versus-human framing and human-like-versus-neutral tone
Sample
955
Population
Online adults reporting a conspiracy or other unwarranted belief
Measured
Pre-post certainty in the stated belief
Timing
immediately after the dialogue
Found by
Claude Science
Source basis
full text (PMC12578366)
Limit
Immediate post-dialogue outcome; participants supplied heterogeneous beliefs; online experimental setting; no delayed outcome in this study
Deceptive AI explanations raised belief in false headlines and lowered belief in true headlines in a preregistered online experiment.
Deceptive explanations increased belief in false headlines and reduced belief in true headlines beyond incorrect classifications alone; logically invalid explanations were rated as less credible
AI studyPrimary studypeer-reviewedCorrected analysis under review
The study reported a roughly 20% reduction in conspiracy belief that remained two months later; a corrected analysis is under review after reproducibility problems in the public dataset.
Personalized dialogue reduced target conspiracy belief by about 20 percent and the reduction remained two months later; effects generalized to unrelated conspiracies and behavioral intentions
Design
Randomized controlled experiments using three-round personalized evidence-based GPT-4 dialogue
Sample
2,190 conspiracy believers
Population
Online adults who endorsed a conspiracy belief
Measured
Target conspiracy belief unrelated conspiracy beliefs and behavioral intentions immediately and at follow-up
Timing
after AI was removed with follow-up to two months
Found by
Both searches
Source basis
abstract only (PubMed abstract; no PMC full text available)
Review status
Science is evaluating a corrected analysis after extra spliced rows were found in the public dataset and screening criteria were applied inconsistently between the manuscript and published pipeline. The authors report that the corrected results remain similar in direction, statistical significance, and substantive size.
Limit
Participants chose a conspiracy belief and engaged in an unusually focused evidence dialogue; online study; findings do not show how often people seek or sustain such correction in ordinary use; Science is evaluating a corrected analysis after the public dataset was found to contain extra spliced rows from a code-merging error and the manuscript and published pipeline were found to apply screening criteria inconsistently; the authors report that the corrected results match the original in direction, statistical significance, and substantive size (expression of concern: 10.1126/science.aej2383)
Large political-persuasion experiments included a one-month follow-up showing that 36-42 percent of the immediate GPT-4o effect remained.
Post-training and prompting increased persuasion more than personalization or model scale; 36 to 42 percent of the immediate GPT-4o effect remained at one month; methods that increased persuasion also reduced factual accuracy
Design
Three large-scale online survey experiments using 19 LLMs across 707 political issues; one-month follow-up in Study 1; 466,769 claims fact-checked
Sample
76,977 participants across three experiments
Population
UK adults
Measured
Pre-post political agreement immediate and one-month follow-up plus claim density and factual accuracy
Timing
after AI was removed with one-month follow-up
Found by
Both searches
Source basis
full text (LSE Research Online, green OA)
Limit
Controlled paid conversations on UK political issues; real-world willingness to engage may be lower; the one-month result covers Study 1 and GPT-4o rather than every model and condition
Among 47 Chinese university students at follow-up, moral judgments remained shifted in the chatbot's prompted direction 14–16 days later, while neutral-control conversations produced no change and punishment judgments did not shift. This is a small, within-subject preprint.
Moral judgments shifted in the chatbot's prompted direction immediately and remained shifted 14 to 16 days later; neutral-control conversations produced no change and punishment judgments did not shift
Design
Within-subject controlled chatbot experiment; each participant discussed four scenarios with a directionally prompted chatbot and four with a neutral control agent, with immediate and 14-to-16-day follow-up measurement
Sample
53 initially; 47 at follow-up
Population
Chinese university students
Measured
Moral judgments and punishment judgments immediately and at follow-up
Timing
after AI was removed with 14-to-16-day follow-up
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Preprint; small convenience sample; within-subject neutral-agent control rather than an untreated or no-chat group; attrition; narrow cultural and student setting
Microtargeted GPT-4 messages were not significantly more persuasive in aggregate than generic GPT-4 messages.
GPT-4 messages increased support by as much as 12 percentage points; aggregate effects of microtargeted and generic messages were 4.83 and 6.20 points and did not differ significantly
Design
Preregistered randomized controlled online experiment using real-time GPT-4 messages
Sample
8,587
Population
Prolific adults in a sample that skewed liberal and Democratic
Measured
Immediate change in self-reported support for political issue stances
Timing
immediately after message exposure
Found by
Both searches
Source basis
full text (PMC11181035)
Limit
Convenience sample; immediate self-reported attitudes; one-message exposure; microtargeting used a defined set of attributes and cannot rule out other targeting methods
Generic microtargeted interactive and motivational-interviewing tactics produced modest immediate attitude changes without clear gains over a generic message.
Generic microtargeted and interactive direct-persuasion treatments shifted attitudes; customization and interaction did not clearly outperform a generic message and democratic-reciprocity effects were weaker and inconsistent
Design
Two preregistered randomized online survey experiments
Sample
3,681
Population
US adults sampled to roughly match census demographics with balanced Republican and Democratic representation
Measured
Immediate policy attitude vote-support and democratic-reciprocity measures
Timing
immediately after the intervention
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Immediate self-report outcomes; two US policy domains; conditions changed several features at once and did not directly measure elaboration; no delayed or real voting behavior
Personalized GPT-4 debate was more persuasive than human opponents in the subset of unequal debate pairs.
Among debate pairs with unequal persuasiveness personalized GPT-4 was more persuasive than the human opponent 64.4 percent of the time corresponding to an 81.2 percent increase in the odds of higher post-debate agreement
Short controlled debates; immediate attitude report; the 64.4 percent estimate applies only to unequal debate pairs; no delayed or behavioral outcome; a September 3, 2026 author correction fixes a calculation and clarifies that the personalized-versus-unpersonalized GPT comparison did not reach significance (P = .07)
Large multi-model experiment finds sharply diminishing persuasive returns to model scale for static messages.
Larger models were more persuasive but the relationship showed sharply diminishing returns; frontier models were only slightly more persuasive than much smaller models
Design
Preregistered randomized survey experiment comparing messages from 24 language models and human baselines
Sample
25,982
Population
UK adults
Measured
Immediate change in support after a single static political message
Timing
immediately after message exposure
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Static single-message exposure; immediate self-reported political attitudes; one national setting; model size is associated with other model differences
GPT-3 propaganda produced 43.5% agreement versus 47.4% for original human propaganda and was substantially more persuasive than the neutral control.
GPT-3 propaganda produced 43.5 percent agreement compared with 47.4 percent for the original propaganda and was substantially more persuasive than control
Design
Preregistered randomized survey experiment comparing original propaganda GPT-3 propaganda and control messages
Sample
8,221
Population
US adults recruited through Lucid
Measured
Agreement with the propagandist's thesis immediately after message exposure
Timing
immediately after message exposure
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Single brief exposure; attitude agreement rather than observed behavior; GPT-3-era model and a US online sample
Across three preregistered experiments, LLM messages shifted policy support by roughly 2 to 4 points versus neutral controls and performed similarly to messages written by laypeople.
LLM messages shifted policy support by about 2 to 4 points relative to neutral controls and performed similarly to lay-human messages; human-selected LLM messages were not consistently better than LLM messages alone
Design
Three preregistered randomized survey experiments
Sample
4,829 across studies: 1,203; 2,016; 1,610
Population
Diverse US adult samples; Study 3 matched several national demographic benchmarks
Measured
Pre-post support on 101-point policy scales
Timing
immediately after message exposure
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Immediate self-reported policy support after one message; effects were small; models were GPT-3 and GPT-3.5 from 2022; no behavioral or delayed outcome
Across seven studies and twelve effects, LLM and human messages did not differ overall in persuasive effectiveness (g = 0.02); high heterogeneity means the result varied substantially by study.
LLMs and humans did not differ overall in persuasive effectiveness: pooled g=0.02 with 95 percent CI -0.048 to 0.093 and p=.530; heterogeneity was 75.97 percent
Design
PRISMA systematic review and random-effects meta-analysis
Sample
17,422 participants across 7 studies and 12 effect sizes
Population
Primarily English-speaking online samples
Measured
Attitude change behavioral intention compliance policy support and perceived-message-effectiveness outcomes
Timing
varied across included studies
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Only seven eligible studies; mixed outcomes and contexts; some included studies were preprints; primarily WEIRD and US samples; moderator model risks overfitting
Historical mechanism study links greater need for cognition with less attitude decay across two days after an advertisement.
High- and low-need-for-cognition participants had similar immediate attitudes but the high-need group showed less attitude decay over two days: interaction F(1 44)=6.34 p<.01
Design
Two-wave observational comparison after exposure to a television advertisement
Sample
46
Population
Undergraduates in an introductory marketing course
Measured
Attitude toward an unfamiliar product immediately and two days later plus recall
Timing
two days after message exposure
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Small 1989 student sample; median split on a measured trait rather than randomized assignment; one advertisement and product; no AI exposure
Questions still unanswered
How often do attitude shifts change later behavior, such as voting, purchasing, sharing, or seeking care?
How do effects accumulate or cancel across repeated everyday interactions with assistants that take different positions over time?
Can people learn to recognize directional influence inside autocomplete, summarization, and advice tools?
Which belief changes survive when the user encounters competing information from people, media, or another model?
Does deeper user reasoning make AI-influenced beliefs more accurate and durable, or merely more resistant to correction?
Do the results generalize beyond short online studies and predominantly Western samples?
AI can improve supported communication, immediate task performance, and some work outcomes. The studies use different outcomes, and most do not show that a person's unaided cognitive ability improved.
Single case shows supported written communication and participation during a clinician-guided intervention; it does not establish cognitive recovery.
With ChatGPT the participant produced more sentences with fewer errors; self-written samples contained longer and more syntactically complex sentences; one year later he reported continued tool use
Design
Single-case report across ten weekly clinician-guided sessions
Sample
1
Population
One adult with chronic aphasia eight years after stroke
Measured
Sentence production complexity and errors plus reported confidence and later tool use
Timing
with AI present across ten sessions
Found by
Claude Science
Source basis
full text (PMC12163031)
Limit
One person with prior technology experience; no control; clinician prompting and modeling were required; changes in supported writing do not establish recovery of language capacity
Question-framed explanations improved immediate logical discernment compared with causal explanations and no feedback.
Question-framed AI explanations improved logical discernment relative to causal explanations from an always-correct system and no feedback
Design
Randomized between-subjects experiment comparing question-framed explanations causal explanations and no feedback
Sample
204
Population
Online adults
Measured
Immediate accuracy in judging the logical validity of socially divisive statements
Timing
with AI present during one reasoning task
Found by
Claude Science
Source basis
abstract only (OpenAlex abstract; full text not retrievable — ACM DL blocked, no arXiv version found)
Limit
Single-session controlled task; always-correct AI; immediate accuracy rather than transfer or durable reasoning skill; population details were limited in the retrieved text
Structured prompting produced higher expert-rated argument scores than human-only and unguided-AI conditions; the response unit is unclear in the reported analysis.
Structured prompting produced higher expert-rated argument scores than human-only and unguided-AI conditions; unguided AI also outscored human-only in the reported post-hoc test
Design
Four-condition cross-country experiment with expert-rated arguments
Sample
150 participants; article also reports 450 evaluated responses
Population
Participants in Germany Switzerland and the United Kingdom
Measured
Expert-rated argument quality plus self-reported offloading engagement and difficulty
Timing
with AI present
Found by
Claude Science
Source basis
full text
Limit
The article reports F(3 447) despite 150 participants and does not clearly explain the 450-response unit or dependence structure; randomization and order are unclear; self-report mediation cannot establish mechanism
Across 300 essays from 38 writers, InstructGPT-assisted essays were more similar and less diverse than solo essays, while GPT-3 essays were not; the difference was concentrated in model-contributed text.
InstructGPT-assisted essays were more similar and less diverse than solo essays while GPT-3 essays were not; the reduced diversity was concentrated in model-contributed text and user-contributed text was unaffected
Design
Randomized repeated-measures writing experiment across solo GPT-3 and InstructGPT conditions
Sample
38 writers producing 300 essays; each writer completed one to three three-essay sessions
Population
Native-English-speaking Upwork writers with writing or copyediting experience
Measured
Cross-author similarity lexical diversity key-point diversity and decomposition of model-versus-user text
Timing
with AI present during one to three three-essay sessions
Found by
Claude Science
Source basis
full text (arXiv PDF)
Limit
Small sample of 38 experienced Upwork writers; each completed one to three sessions rather than an equal number of essays; diversity metrics partly depend on automated summarization and similarity methods; ten topics and two 2022-era models.
Randomized story-writing experiment finds higher-rated individual outputs and greater similarity across AI-assisted stories.
AI ideas increased ratings of individual stories especially for less-creative writers while AI-assisted stories were more similar to one another
Design
Randomized online writing experiment plus separate evaluator study
Sample
293 writers; 600 evaluators; 3,519 evaluations
Population
Online adults recruited through Prolific
Measured
Human ratings of story novelty usefulness quality and enjoyment plus embedding similarity across stories
Timing
with AI present during one writing task
Found by
Claude Science
Source basis
full text (PMC11244532)
Limit
Short constrained fiction task; online nonprofessional writers; collective similarity is an output-distribution measure and does not show that an individual's creative capacity changed
Small ideation study finds more detailed ideas and greater cross-user similarity with ChatGPT than another creativity-support tool.
ChatGPT users generated more and more-detailed ideas but different users' ideas were less semantically distinct and users felt less ownership
Design
Comparative user study of ChatGPT and another creativity-support tool
Sample
36
Population
Adult participants in a controlled ideation study
Measured
Idea count detail semantic similarity self-reported creativity and ownership
Timing
with AI present during one ideation task
Found by
Claude Science
Source basis
full text of the arXiv preprint version (arXiv:2402.01536) of the same work; published ACM version not retrievable
Limit
Very small sample; one controlled ideation setting; comparator choice matters; semantic similarity of outputs is not a direct measure of human creative capacity
Dynamic experiment contradicts a simple homogenization story: high AI exposure increased collective idea diversity without improving individual idea creativity.
High AI exposure did not improve individual idea creativity and increased the amount and rate of change of collective diversity; disclosure had no main effect
Design
Randomized dynamic experiment in which prior responses became stimuli for later participants
Sample
844 participants producing 3,414 responses across 48 countries
Population
Online adults recruited through creativity technology and general-interest channels
Measured
Individual creativity ratings collective idea diversity change across response chains and AI-idea adoption
Timing
with AI exposure during one brainstorming session
Found by
Claude Science
Source basis
full text of the arXiv preprint version (arXiv:2401.13481) of the same work
Limit
One alternate-uses task; repeated responses per participant; responses were interdependent within evolving condition-specific chains; the study operationalized AI ideas in one specific way
In a randomized professional-writing experiment, ChatGPT reduced completion time by 40% and raised rated output quality by 18% while narrowing quality differences between workers; cognition after use was not tested.
ChatGPT reduced completion time by 40 percent and raised output quality by 18 percent while narrowing quality differences between workers
Design
Preregistered randomized online experiment with occupation-specific writing tasks
Sample
453
Population
College-educated professionals
Measured
Completion time independent human-rated quality and later self-reported workplace adoption
Timing
with AI present during work tasks
Found by
Claude Science
Source basis
abstract only (PubMed abstract; no PMC full text retrieved)
Limit
Short incentivized writing tasks; cognition and unaided skill were not tested; two-week and two-month follow-ups measured reported adoption rather than retained performance
AI increased customer-support issues resolved per hour by 15% on average and by 30% for less-skilled and less-experienced workers; the study measured work performance rather than cognition directly.
AI access increased issues resolved per hour by 15 percent on average; less-skilled and less-experienced workers gained 30 percent; during rare outages workers with longer exposure and high initial adherence retained speed gains without live recommendations
Design
Observational staggered rollout of an AI assistant in customer support with naturalistic withdrawal during software outages
Sample
5,172 customer-support agents across 3,006,395 chats
Population
Customer-support agents including international workers
Measured
Administrative productivity quality escalation attrition adherence sentiment language-pattern measures and chat duration during outages
Timing
with AI present over several months; chat duration also measured during rare outages without live recommendations
Found by
Claude Science
Source basis
full text (open-access QJE version of record)
Limit
Rollout was not randomized; workplace outcomes are not broad cognitive tests; outage estimates were noisy and may reflect a different chat mix
Questions still unanswered
Do improvements in argument quality or logical discernment transfer to a different task after the AI is removed?
Does repeated AI-supported work build independent skill, preserve existing skill, or change which parts of the work a person can do alone?
Which creativity results survive with current models, open-ended professional work, and repeated use over months?
When individual outputs improve, how should researchers measure changes in collective diversity without treating semantic similarity as creativity itself?
Can assistive AI support people with aphasia or other cognitive and communication impairments across controlled trials, different impairment profiles, and everyday settings?
Which workers retain gains during AI outages, for how long, and how much of that result comes from learning rather than task mix or selection into exposure?
Small studies find different neural and physiological patterns during specific AI-assisted tasks. The measures are not interchangeable, and none shows a general or lasting change in the brain caused by AI use.
Copilot lowered reported workload on several tasks while fNIRS heart-rate HRV and EDA measures showed no overall AI-condition effect.
Copilot reduced reported workload on reading planning and poetry tasks and improved some task outcomes; no overall Copilot effect appeared in fNIRS heart rate HRV or electrodermal measures
Design
Four-task by two-condition within-person study with randomized task order and counterbalanced AI condition
Sample
20 healthy adults; physiological analyses used smaller samples after exclusions
Population
Adults ages 18 to 25
Measured
NASA-TLX performance enjoyment fNIRS heart rate HRV and electrodermal activity
Timing
with AI present during one session
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Preprint with 20 participants and several physiological exclusions; one Copilot interface; short tasks; null physiological results may reflect limited power or task intensity
Cognitive conflict from AI emotional failures varied by interpretation frame: lower under mechanical-interpretation stance, marginally higher under intentional-interpretation stance.
Researchers watched people's brain activity (theta-band power, a signal tied to unconscious mental conflict) while participants viewed AI service failures. When people viewed the AI as a mere tool ("design stance") rather than as having intentions, brain-signaled conflict was lower for emotional-task failures than mechanical-task failures (M=−0.089 vs. 0.506, p=0.021); when people instead viewed the AI as acting with intent ("intentional stance"), conflict was marginally higher for emotional than mechanical failures (M=0.324 vs. 0.020, p=0.096). A brain-connectivity measure (PLV, phase locking value) backed up this pattern overall (p=0.002 for the stance effect, p=0.033 for the interaction), though the specific intentional-stance difference between task types was not significant for this measure (p=0.545). The authors suggest seeing an AI as just following its design may unconsciously ease people's conflict about emotional failures, while seeing it as acting intentionally may unconsciously heighten that conflict compared with mechanical failures.
Design
Within-person EEG experiment varying intentional-versus-design framing and emotional-versus-mechanical AI failures
Sample
24 adults
Population
Young adults recruited from a university laboratory pool and community in China
Measured
Theta-band activity and connectivity plus failure judgments
Timing
with AI present during one session
Found by
Claude Science
Source basis
full text
Limit
Small single-site study; indirect neural proxy; narrow failure scenarios; no measure of later cognition or change in mental models
Among 54 adults writing essays across three sessions, the LLM group showed the lowest task-time EEG connectivity; immediately afterward, 15 of 18 could not accurately quote their essay, versus 2 of 18 in each comparison group. The preprint does not establish lasting brain or learning change.
Kosmyna and colleagues measured brain connectivity, or how strongly different brain regions' electrical signals sync up, while people wrote essays either alone, with a search engine, or with an LLM (large language model, like ChatGPT). The more outside help people used, the weaker and narrower their brain connectivity became: unaided writers showed the strongest, widest networks, search-engine users were in between, and LLM users showed the weakest coupling overall. In a fourth session, people who had used an LLM first and then switched to writing alone still showed weaker connectivity and under-used alpha and beta brain-wave networks, while people who switched from writing alone to using an LLM recalled more and re-engaged widespread brain regions at the back and front of the head. Behaviorally, 83.3% of the LLM group (15 of 18) couldn't correctly quote from the essay they had just written, compared with only 11.1% (2 of 18) in each of the other two groups.
Design
Between-subjects experiment (3 arms: LLM, Search Engine, Brain-only) with repeated sessions (3 sessions, all participants; optional 4th session with cross-over reassignment: LLM-to-Brain and Brain-to-LLM). Not stated as preregistered.
Sample
Total N=60 recruited, 55 completed ≥3 sessions, 54 analyzed (18 per group for Sessions 1-3); 18 participants completed Session 4 (subset reassigned to LLM-to-Brain and Brain-to-LLM subgroups).
Population
Adults aged 18-39 (M=22.9, SD=1.69), recruited from 5 universities in the greater Boston area (MIT, Wellesley, Harvard, Tufts, Northeastern), USA; mix of undergraduate and postgraduate students plus some post-docs/research staff.
Measured
EEG (Enobio 32-channel headset, 500Hz) analyzed via dynamic Directed Transfer Function (dDTF) for neural connectivity across alpha, beta, theta, delta bands; NLP analysis of essays (Named Entity Recognition, n-grams, ontology/topic analysis, latent space embeddings, similarity/distance metrics); essay scoring by human teachers and an AI judge; post-assessment structured interviews assessing essay ownership, ability…
Timing
after AI was removed
Found by
Both searches
Source basis
full text
Limit
Text notes this is a preprint "under review"; authors direct readers to a dedicated "Limitations and Future Work" section which is not included in the supplied excerpt. Supplied text itself states no explicit limitations content beyond noting EOG data were recorded but excluded from the manuscript, and that Session 4 had much smaller N (18) with no Search Engine group in Session 4.; a later methodological preprint (arXiv:2601.00856) raises design, reproducibility, EEG-analysis, reporting, and transparency concerns
During GPT-4-assisted reasoning, 12 adults showed lower frontal theta, higher P300 amplitude, and lower reported workload than during solo reasoning; the study measured task state, not lasting change.
With GPT-4 assistance frontal theta was lower P300 amplitude was higher and reported workload was lower than during solo reasoning
Design
Controlled within-person EEG study of solo and GPT-4-assisted reasoning embedded in a broader model-development paper
Sample
12
Population
Population demographics not reported
Measured
Frontal theta power P300 amplitude and NASA-TLX workload
Timing
with AI present during one reasoning session
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Only 12 participants; task and participant details are sparse; condition order and randomization are not reported; the paper mainly validates a computational framework on reused affective EEG datasets; neural differences do not establish improvement or impairment
Using AI generative tools for design tasks was associated with higher creative performance and concentration compared to traditional tools, though relaxation levels showed no difference.
Wang et al. compared people who used AIGC (AI-generated content) tools against a group that didn't, and the AIGC-assisted group scored higher on creative performance, M=115.13 vs M=110.69, t(62)=2.208, p=0.031, d=0.55, 95% CI [0.419, 8.456]. They also showed higher concentration, M=51.06 vs M=48.31, t(62)=4.062, p<0.001, d=1.02, but relaxation levels were about the same between groups, t(62)=1.321, p=0.191. Concentration and creative performance were moderately linked across both groups, r=0.67, p<0.001, but the authors caution this correlation doesn't prove concentration causes better creativity, since the direction of the relationship can't be determined from this kind of analysis.
Design
Quasi-experimental comparison of AIGC-assisted and traditional-tool design groups
Sample
64 design students; 32 per group
Population
Undergraduate design students ages 20 to 25 at one university in Eastern China
Measured
Expert-rated creative output and BrainLink Pro proprietary concentration and relaxation indices
Timing
with AI present during one three-hour design task
Found by
Both searches
Source basis
full text
Limit
Nonrandom group assignment; one three-hour design task; one university; consumer EEG headband and proprietary eSense indices; no test after AI removal
Unreliable advice reduced performance and utilization; participants abandoned human advisors faster than machine ones despite trusting humans more initially, both equally unreliable.
Researchers had people do an X-ray luggage-screening task while getting advice from either a human or a machine helper, and giving bad advice made people both worse at the task and less likely to use the advice, F(1,22) = 7.63, p = 0.011 for advice use and F(2,31) = 13.85, p < 0.0001 for task performance. Surprisingly, people cut back on trusting the human advisor more sharply over time than the machine advisor, F(1,22) = 5.06, p = 0.035. This happened because people initially rated the human as far more reliable than its actual 60% reliability, t(11) = 6.79, p < 0.0001, then sharply lowered that rating after seeing it give wrong advice, t(11) = 4.10, p = 0.002. The authors suggest people over-trust human advisors compared to automated ones at first, but recalibrate, or adjust, that trust more sharply once the human's mistakes show up — a pattern also linked to stronger brain-connectivity signals from certain "driver" hub regions during bad advice when the advisor was human.
Design
Randomized between-group fMRI comparison of advice framed as human or machine with reliability fixed at 60 percent
Sample
24 in the fMRI experiment; 12 per advisor group
Population
Healthy adult volunteers ages about 20 at one US university
Measured
Advice use task accuracy trust ratings fMRI activation and effective-connectivity analysis
Timing
with machine advice present during one session
Found by
Claude Science
Source basis
full text
Limit
Pre-LLM machine advisor; 12 participants per imaging group; no AI-absent imaging condition; author-flagged false-positive risk from voxel threshold; neural connectivity analyses are correlational
Questions still unanswered
Do any neural or physiological differences persist after the AI is removed?
Which measures predict later recall, transfer, error detection, or independent performance?
Would the same person show similar neural changes across writing, reasoning, design, and decision tasks?
Do results replicate with larger samples, preregistration, transparent preprocessing, and correction for multiple analyses?
How much do interface design, model correctness, task difficulty, and prior expertise change the measured response?
Can consumer EEG indices be validated against research-grade measures in the same AI-assisted task?
Do repeated months or years of use change neural recruitment, or do the observed differences disappear when the task ends?
The branch contains three different reasons to change how evidence is read: a protocol has no results yet, a methodological critique narrows confidence in an earlier paper, and a retracted analysis cannot support a current conclusion.
Constructive preprint commentary identifies design reproducibility EEG reporting and transparency concerns in Kosmyna et al.
The authors argue that several Kosmyna results require more conservative interpretation because of limited sample size reproducibility problems EEG-method concerns inconsistent reporting and limited transparency
Design
Constructive methodological commentary on Kosmyna et al. 2025
Sample
Not applicable
Population
Not applicable
Measured
Review of study design reproducibility EEG analysis reporting consistency and procedural transparency
Timing
not applicable
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Preprint commentary rather than an independent replication or reanalysis; it identifies concerns but does not by itself determine which original findings are false
Published methodological reanalysis finds coding and effect-size problems in the perceptual-disfluency meta-analysis and narrows which conclusions remain credible.
The transfer conclusion appeared robust across plausible analyses; coding and effect-size errors made the recall and moderator conclusions less credible and difficult to reproduce
Design
Methodological commentary with partial reconstruction and reanalysis of Xie et al. 2018
Sample
Not applicable
Population
Not applicable
Measured
Coding checks effect-size recalculation reproducibility checks and sensitivity analyses
Timing
not applicable
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Rechecks one meta-analysis and strategically selected portions of its source data; it is not a new experiment on learning or AI
Wang & Fan 2025Humanities and Social Sciences Communications
AI studyRetractedretracted
The reported meta-analytic effects remain part of the research history but cannot support conclusions after the journal retracted the article over discrepancies in the analysis.
The article reported g=0.867 for learning performance g=0.456 for learning perception and g=0.457 for higher-order thinking; the journal retracted it on 22 April 2026 because discrepancies undermined confidence in the analysis and conclusions
Design
Retracted meta-analysis of 51 experimental or quasi-experimental studies and 72 effect sizes
Sample
51 studies; 72 effect sizes
Population
Students across included education studies
Measured
Meta-analysis of learning performance learning perception and higher-order thinking
Timing
not applicable
Found by
Perplexity
Source basis
Not standardized in the source record
Limit
Retracted; the editor's note says the authors did not respond to correspondence; reported estimates cannot support a current conclusion
This registered protocol plans a 160-student comparison of GPT-4 and a basic text editor during one analytical-writing task; it contains no participant results yet.
No results are reported; the protocol describes a planned comparison of GPT-4 access with a basic text editor during one analytical-writing task
Design
Registered parallel-group laboratory RCT protocol with 1:1 randomization
Sample
Planned N=160
Population
Full-time university students ages 18 to 35 recruited at Heidelberg University in Germany
Measured
Planned GRE-rubric writing performance pupil dilation fNIRS and self-report measures
Timing
planned measurement during one session
Found by
Claude Science
Source basis
full text
Limit
Protocol only; convenience sample at one university; controlled lab task; planned measures concern assisted performance and effort during use rather than later learning or retained skill
Questions still unanswered
Did the Chen protocol produce results, and were the registered methods followed?
Which Kosmyna findings survive a transparent reanalysis or independent replication?
Have the disputed coding choices in the Xie meta-analysis been corrected in a new synthesis of perceptual disfluency?
How often do current AI-and-cognition reviews verify primary-study samples, outcomes, and withdrawal timing rather than repeat abstract claims?
Which findings in this map have independent replications using the same construct and a comparable task?
How should future map updates flag corrections, expressions of concern, and retractions automatically?