Conversation

Notices

Embed this notice
Henrik Schönemann (lavaeolus@fedihum.org)'s status on Sunday, 15-Dec-2024 02:21:15 JST Henrik Schönemann

"#AI" as Artificial Intelligence is not real. But the stuff being called "AI" most certainly is.
Simply telling people "AI is not real" is not helpful in the slightest. You're telling them: "Your perception of reality is wrong", but the machine does things that are 'indistinguishable from magic' (Hi Clarke!) to them.
We won't convince people by telling them "Your god isn't real! Follow me!" (I'm exaggerating), but by empowering them to emerge from their self-imposed immaturity.

In conversation about 2 months ago from fedihum.org permalink
- Embed this notice
  Thanasis Kinias (tkinias@historians.social)'s status on Sunday, 15-Dec-2024 02:21:11 JST Thanasis Kinias
  in reply to
  
  @lavaeolus
  I’m thinking more informally—like “talking to family over Xmas dinner” where I can’t assign them academic reading 😉
  What I often get when pointing out “the limitations, the weirdness, the brokenness” is that it’s early days and these problems will go away as the tech matures (a perspective, of course, predicated on a failure to understand the tech).
  
  In conversation about 2 months ago permalink
- Embed this notice
  alcinnz (alcinnz@floss.social)'s status on Sunday, 15-Dec-2024 02:21:11 JST alcinnz
  in reply to
  - Thanasis Kinias
  @tkinias @lavaeolus It drives me nuts arguing that it isn't early days. LLMs are just machine learning (scaled ludicrously up), & we've had that since the 1990s. Or '60s!
  
  In conversation about 2 months ago permalink
- Embed this notice
  Henrik Schönemann (lavaeolus@fedihum.org)'s status on Sunday, 15-Dec-2024 02:21:13 JST Henrik Schönemann
  in reply to
  - Thanasis Kinias
  @tkinias by showing them - not telling, but showing.
  Show them the limitations, the weirdness, the brokenness.
  E.g.: Jacomy, Mathieu, and Erik Borra. 2024. “Measuring LLM Self-Consistency. Unknown Unknowns in Knowing Machines.” Sociologica 18 (2): 25–65. https://doi.org/10.6092/ISSN.1971-8853/19488.
  In conversation about 2 months ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: sociologica.unibo.it
    
    Measuring LLM Self-consistency: Unknown Unknowns in Knowing Machines
    
    This essay critically examines some limitations and misconceptions of Large Language Models (LLMs) in relation to knowledge and self-knowledge, particularly in the context of social sciences and humanities (SSH) research. Using an experimental approach, we evaluate the self-consistency of LLM responses by introducing variations in prompts during knowledge retrieval tasks. Our results indicate that self-consistency tends to align with correct responses, yet errors persist, questioning the reliability of LLMs as “knowing” agents. Drawing on epistemological frameworks, we argue that LLMs exhibit the capacity to know only when random factors, or epistemic luck, can be excluded, yet they lack self-awareness of their inconsistencies. Whereas human ignorance often involves many “known unknowns”, LLMs exhibit a form of ignorance manifested through inconsistency, where the ignorance remains a complete “unknown unknown”. LLMs always “assume” they “know”. We repurpose these insights into a pedagogical experiment, encouraging SSH scholars and students to critically engage with LLMs in educational settings. We propose a hands-on approach based on critical technical practice, aiming to balance the practical utility with an informed understanding of their limitations. This approach equips researchers with the skills to use LLMs effectively while promoting a deeper understanding of their operational principles and epistemic constraints.
- Embed this notice
  Thanasis Kinias (tkinias@historians.social)'s status on Sunday, 15-Dec-2024 02:21:14 JST Thanasis Kinias
  in reply to
  
  @lavaeolus
  so... how *do* we explain to laypeople that “AI” (which for most people now means ChatGPT and other LLMs) doesn’t do what the marketing says it does? (different, if subtly so, from “it’s not real”)
  
  In conversation about 2 months ago permalink
- Embed this notice
  Henrik Schönemann (lavaeolus@fedihum.org)'s status on Sunday, 15-Dec-2024 02:50:59 JST Henrik Schönemann
  in reply to
  - alcinnz
  - Thanasis Kinias
  @alcinnz and they are only talking about language models, Computational Linguistics etc. have been doing stuff for much longer
  @tkinias
  
  In conversation about 2 months ago permalink
- Embed this notice
  Henrik Schönemann (lavaeolus@fedihum.org)'s status on Sunday, 15-Dec-2024 02:51:00 JST Henrik Schönemann
  in reply to
  - alcinnz
  - Thanasis Kinias
  @alcinnz 100% yes!
  I love using this graph from Zhao, Wayne Xin, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, et al. 2024. ‘A Survey of Large Language Models’. arXiv. https://doi.org/10.48550/arXiv.2303.18223.
  @tkinias
  In conversation about 2 months ago permalink
  Attachments
  1. Untitled attachment
    https://fedihum.s3proxy.de/media_attachments/files/113/652/431/577/364/400/original/357d582d219f857c.png
  2. Domain not in remote thumbnail source whitelist: arxiv.org
    
    A Survey of Large Language Models
    
    Language is essentially a complex, intricate system of human expressions governed by grammatical rules. It poses a significant challenge to develop capable AI algorithms for comprehending and grasping a language. As a major approach, language modeling has been widely studied for language understanding and generation in the past two decades, evolving from statistical language models to neural language models. Recently, pre-trained language models (PLMs) have been proposed by pre-training Transformer models over large-scale corpora, showing strong capabilities in solving various NLP tasks. Since researchers have found that model scaling can lead to performance improvement, they further study the scaling effect by increasing the model size to an even larger size. Interestingly, when the parameter scale exceeds a certain level, these enlarged language models not only achieve a significant performance improvement but also show some special abilities that are not present in small-scale language models. To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size. Recently, the research on LLMs has been largely advanced by both academia and industry, and a remarkable progress is the launch of ChatGPT, which has attracted widespread attention from society. The technical evolution of LLMs has been making an important impact on the entire AI community, which would revolutionize the way how we develop and use AI algorithms. In this survey, we review the recent advances of LLMs by introducing the background, key findings, and mainstream techniques. In particular, we focus on four major aspects of LLMs, namely pre-training, adaptation tuning, utilization, and capacity evaluation. Besides, we also summarize the available resources for developing LLMs and discuss the remaining issues for future directions.
  alcinnz repeated this.
- Embed this notice
  alcinnz (alcinnz@floss.social)'s status on Sunday, 15-Dec-2024 02:52:55 JST alcinnz
  in reply to
  - Thanasis Kinias
  @lavaeolus I had neural nets specifically in mind. If I recall correctly we were playing with perceptrons in the '60s, got disappointed, & resuscitated the ideas at a larger scale the '90s.
  @tkinias
  
  In conversation about 2 months ago permalink

Public

Conversation

Notices

Feeds