Notices by nataliyakosmyna (nataliyakosmyna@mastodon.social)

@n_dimension @cwebber it is a preprint indeed! My first one, to be honest (two other papers on arxiv under my profile were not submitted by myself), and I see your points about the preprints, I also prefer the papers to be peer-reviewed first! But I felt like in 6-8 months (best case!), when the paper will undergo the peer reviews, some policymaker will already decide to fully vote in favor of LLM to be integrated in all kindergartens across the country.

@cwebber @djuber thank you! I do love to hear from the “source” as well! We are tired but proud indeed!

𝐈𝐕. 𝐁𝐞𝐡𝐚𝐯𝐢𝐨𝐫𝐚𝐥 𝐚𝐧𝐝 𝐂𝐨𝐠𝐧𝐢𝐭𝐢𝐯𝐞 𝐄𝐧𝐠𝐚𝐠𝐞𝐦𝐞𝐧𝐭
- Quoting Ability: LLM users failed to quote accurately, while Brain-only participants showed robust recall and quoting skills.
- Ownership: Brain-only group claimed full ownership of their work; LLM users expressed either no ownership or partial ownership.
- Critical Thinking: Brain-only participants cared more about 𝘸𝘩𝘢𝘵 and 𝘸𝘩𝘺 they wrote; LLM users focused on 𝘩𝘰𝘸.

- Cognitive Debt: Repeated LLM use led to shallow content repetition and reduced critical engagement. This suggests a buildup of "cognitive debt", deferring mental effort at the cost of long-term cognitive depth.

Results for Brain-to-LLM participants suggest that timing of AI tool introduction following initial self-driven effort may enhance engagement and neural integration.

𝑺𝒆𝒔𝒔𝒊𝒐𝒏 4 𝑹𝒆𝒔𝒖𝒍𝒕𝒔:
- LLM-to-Brain (🤖🤖🤖🧠) participants underperformed cognitively with reduced alpha/beta activity and poor content recall.
- Brain-to-LLM (🧠🧠🧠🤖) participants showed strong re-engagement, better memory recall, and efficient tool use.

LLM-to-Brain participants had potential limitations in achieving robust neural synchronization essential for complex cognitive tasks.

𝐈𝐈𝐈: 𝐄𝐄𝐆 𝐀𝐧𝐚𝐥𝐲𝐬𝐢𝐬
Connectivity: Brain-only group showed the highest neural connectivity, especially in alpha, theta, and delta bands. LLM users had the weakest connectivity, up to 55% lower in low-frequency networks. Search Engine group showed high visual cortex engagement, aligned with web-based information gathering.

𝐈𝐈. 𝐄𝐬𝐬𝐚𝐲 𝐒𝐜𝐨𝐫𝐢𝐧𝐠 (𝐓𝐞𝐚𝐜𝐡𝐞𝐫𝐬 𝐯𝐬. 𝐀𝐈 𝐉𝐮𝐝𝐠𝐞)
- Teachers detected patterns typical of AI-generated content and scoring LLM essays lower for originality and structure.
- AI Judge gave consistently higher scores to LLM essays, missing human-recognized stylistic traits.

For 4 months, 54 students were divided into three groups: ChatGPT, Google -ai, and Brain-only. Across 3 sessions, each wrote essays on SAT prompts. In an optional 4th session, participants switched: LLM users used no tools (LLM-to-Brain), and Brain-only group used ChatGPT (Brain-to-LLM).

𝐈. 𝐍𝐋𝐏 𝐚𝐧𝐝 𝐄𝐬𝐬𝐚𝐲 𝐂𝐨𝐧𝐭𝐞𝐧𝐭
- LLM Group: Essays were highly homogeneous within each topic, showing little variation. Participants often relied on the same expressions or ideas.
- Brain-only Group: Diverse and varied approaches across participants and topics.
- Search Engine Group: Essays were shaped by search engine-optimized content; their ontology overlapped with the LLM group but not with the Brain-only group.

𝐍𝐨, 𝐲𝐨𝐮𝐫 𝐛𝐫𝐚𝐢𝐧 𝐝𝐨𝐞𝐬 𝐧𝐨𝐭 𝐩𝐞𝐫𝐟𝐨𝐫𝐦 𝐛𝐞𝐭𝐭𝐞𝐫 𝐚𝐟𝐭𝐞𝐫 𝐋𝐋𝐌 𝐨𝐫 𝐝𝐮𝐫𝐢𝐧𝐠 𝐋𝐋𝐌 𝐮𝐬𝐞.

See our paper for more results: "Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task" : www.brainonllm.com

@cwebber @hailey thank you for mentioning the Search Engine group. I feel seen :-) (I’m the lead author of the paper)

@djuber @cwebber we see you !

@inthehands just as a note, we did not use the language “LLM makes you stupid” in the paper. It is written in an academic manner, though we did add TL;DR section as well as a summary table for an LLM, as the final document is 206 pages (with appendix). But we did show the reduced neural connectivity and a lot of other issues LLM group faced. If you are interested, our summary of the paper is here (authors of this paper): https://www.linkedin.com/posts/nataliekosmina_mit-ai-brain-activity-7340386826504876033-X45W?utm_source=share&utm_medium=member_ios&rcm=ACoAAANkfbABvU568kU63aYOiOdVABVfyyA2Trs

@inthehands we do call this on-going phenomena “cognitive debt”.