Conversation

Notices

1. Embed this notice
   Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social)'s status on Tuesday, 03-Oct-2023 23:24:28 JST Prof. Emily M. Bender(she/her)

   Q for #AcademicFedi, #HCI, #lazyweb: What has been published (if anything, yet) about how lay people understand what ChatGPT is and how it works?

   • clacke likes this.
   • pettter and clacke repeated this.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 03-Oct-2023 23:39:11 JST 翠星石

     in reply to
     @emilymbender I don't have an article, but to summarize, it's pretty much an advanced form of lossy compression for text, running on someone else's server, which one can interface with a web interface that's written in JavaScript which is under a proprietary license.
     
     "OpenAI" has scraped a bunch of text from the internet, most under proprietary terms, some under free terms (but with certain requirements like attribution) and shoved the whole lot through a neural network and the end result is a very energy-hungry language model that accepts question text and vomits out convincing combinations of the input text that matches the question text somewhat, with the output filtered through grammar, spelling and tone checking so it looks very convincing.
     
     By pure chance it gets some things right, but it gets most things wrong due to how much of the input was incorrect and the semi-lossy reproduction nature.
   • Embed this notice
     Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social)'s status on Wednesday, 04-Oct-2023 09:08:21 JST Prof. Emily M. Bender(she/her)

     in reply to

     • john laudun

     @johnlaudun Not really looking for lay literature, but rather academic studies of lay understanding.

     clacke and simsa03 like this.
   • Embed this notice
     john laudun (johnlaudun@hcommons.social)'s status on Wednesday, 04-Oct-2023 09:08:22 JST john laudun

     in reply to

     @emilymbender would you count all the stuff on “prompt engineering” part of the lay literature? This is really an interesting question, getting at what people think about when they think about ChatGPT, AI.

   • Embed this notice
     Eric P (ep_jhu@universeodon.com)'s status on Tuesday, 17-Oct-2023 23:15:59 JST Eric P

     in reply to

     @emilymbender Not specific to ChatGPT, but this Drexel/IBM paper is along those lines: "People’s Perceptions Toward Bias and Related Concepts in Large Language
     Models: A Systematic Review" - https://arxiv.org/pdf/2309.14504.pdf

     clacke likes this.
   • Embed this notice
     john laudun (johnlaudun@hcommons.social)'s status on Wednesday, 18-Oct-2023 00:33:18 JST john laudun

     in reply to

     • john laudun
     • helenbeetham

     @helenbeetham @emilymbender @johnlaudun a grad student just proposed something similar tonight. Any references would be welcome: I’ll pass them along.

     clacke likes this.
   • Embed this notice
     helenbeetham (helenbeetham@mstdn.social)'s status on Wednesday, 18-Oct-2023 00:33:23 JST helenbeetham

     in reply to

     • john laudun

     @emilymbender @johnlaudun not quite what you are looking for but I have been informally collecting examples of 'advice to teachers' and feel there is a need to understand how ppl bcm generative AI 'experts'. Some of the prompts suggest the experts are not even trying out their own recommendations, let alone exploring them in collaboration with students. I cd point u to some positivist studies of student 'perceptions of genAI', treating students as lay ppl and perceptions as measurable.

     clacke likes this.
   • Embed this notice
     Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social)'s status on Wednesday, 18-Oct-2023 00:33:26 JST Prof. Emily M. Bender(she/her)

     in reply to

     • Eric P

     @ep_jhu Thanks!

     clacke likes this.
   • Embed this notice
     Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social)'s status on Wednesday, 18-Oct-2023 00:33:28 JST Prof. Emily M. Bender(she/her)

     in reply to

     • Chung-chieh Shan單中杰

     @ccshan Thanks That's close but doesn't seem to address LLMs/ChatGPT

     clacke likes this.
   • Embed this notice
     Chung-chieh Shan單中杰 (ccshan@woof.group)'s status on Wednesday, 18-Oct-2023 00:33:33 JST Chung-chieh Shan單中杰

     in reply to

     @emilymbender https://doi.org/10.1007/s10639-023-11600-3 seems relevant? Can send to you.

     Kim, K., Kwon, K., Ottenbreit-Leftwich, A. et al. Exploring middle school students’ common naive conceptions of Artificial Intelligence concepts, and the evolution of these ideas. Educ Inf Technol 28, 9827–9854 (2023).

     This study aims to explore the middle schoolers' common naive conceptions of AI and the evolution of these conceptions during an AI summer camp. Data were collected from 14 middle school students

     clacke likes this.
   • Embed this notice
     Colin Rowat (rowat_c@mastodon.social)'s status on Wednesday, 18-Oct-2023 00:33:41 JST Colin Rowat

     in reply to

     @emilymbender I've not seen academic studies, but have been surprised - when speaking to lay audiences - by how well a basic explanation (text prediction + RLHF) gets them to the point where they can answer the questions they identified at the beginning of the sessions: can you trust the answers; is there bias; what sort of questions do they answer well...?

     clacke likes this.
   • Embed this notice
     Prem Kumar Aparanji 👶🤖🐘 (prem_k@mastodon.social)'s status on Wednesday, 18-Oct-2023 00:33:43 JST Prem Kumar Aparanji 👶🤖🐘

     in reply to

     @emilymbender might be helpful: https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged

     clacke likes this.