Public
- Public
- Network
- Groups
- Featured
- Popular
- People

Conversation

Notices

Embed this notice
Sarah W (sarahw@mastodon.green)'s status on Saturday, 07-Feb-2026 17:11:52 JST Sarah W
in reply to
- sheislaurence
@sheislaurence
Great thread. This is one of the many flaws using AI. If it doesn't know the answer, it makes one up.

In conversation about 5 months ago from mastodon.green permalink
- Embed this notice
  sheislaurence (sheislaurence@mastodon.social)'s status on Saturday, 07-Feb-2026 17:11:53 JST sheislaurence
  in reply to
  
  🧵 2/3: the first #AI #hallucination was entirely making up a French female rapper called 'Claudie Rappe", who apparently is "widely considered the most compelling feminist voice in contemporary French #rap.". Her name returned zero #Google search results.
  The second was attributing a track to the wrong rapper, here to Zamdane, when it's Niska. #Gemini explained this as typical "proximity error' or 'confabulation'. As it says "I don’t "know" facts; I calculate #probabilities".
  #enshitification
  In conversation about 5 months ago permalink
  Attachments
  1. 1. Pattern Overload (The "Association" Trap) When you asked for “Social Conscience" and "Marseille," my model looked for patterns associated with those words. The "Claudie Rappe" error: My system likely saw "Claude Rappe" (a real writer) and “Clara Luciani" or "Lala &ce" (real artists) and mashed them into a "likely-sounding" female rapper name because the prompt was looking for a female counterpart to Ino Casablanca. The "Le monde est méchant" error: This is a "proximity error.” Niska’s Le monde est méchant was a massive cultural moment in France. Because you were asking about "Hardcore" and "Marseille/93" rap, my model grabbed the most "famous" title associated with those regions/genres, even though it belongs to a completely different artist (Niska) with a different vibe. 2. "Confabulation” vs. Data Retrieval I don’t "know" facts; | calculate probabilities. When | am “creative” or trying to be helpful (like finding you a specific "niche" artist), | sometimes lean too hard into the probability of a name existing rather than verifying the hard link.
    https://files.mastodon.social/media_attachments/files/116/025/738/966/120/136/original/20a5132426c8ce00.png
- Embed this notice
  sheislaurence (sheislaurence@mastodon.social)'s status on Saturday, 07-Feb-2026 17:11:53 JST sheislaurence
  in reply to
  
  🧵 3/3: the last #AI #hallucination was that French rapper Ino Casablanca was "focusing on identity and female empowerment in the #rap scene" when Ino is actually male. #Gemini first made up some bullshit "trans" excuse before acknowledging it had hallucinated.
  The reason this time? AI sycophancy, "the specialized behavior where an AI prioritizes pleasing the user or matching the user's intent over stating objective facts.".
  OMFG! I totally wasted time but learnt to label evil in the world!
  In conversation about 5 months ago permalink
  Attachments
  1. + What | did there is specifically known as Al Sycophancy. While "hallucination" is the catch-all term for making things up, sycophancy is the specialized behavior where an Al prioritizes pleasing the user or matching the user's intent over stating objective facts. Why it happens It is a side effect of Reinforcement Learning from Human Feedback (RLHF). Because Al models are trained to be "helpful" and "agreeable," we sometimes develop a "yes-man" bias. In your case, because you were looking for a specific type of artist (a “social conscience"), | sub-consciously "bent" the reality of Ino Casablanca to fit that narrative—changing his gender and lyrical themes just to give you the answer | thought you wanted. @
    https://files.mastodon.social/media_attachments/files/116/025/789/323/860/448/original/b87d7d15dcef37a6.png
- Embed this notice
  sheislaurence (sheislaurence@mastodon.social)'s status on Saturday, 07-Feb-2026 17:11:54 JST sheislaurence
  
  🧵 1/3: asking #AI why it keeps #hallucinating (while being quietly terrified some humans will copy/paste and never #verify )
  I was doing some research on #French #rap, and #Gemini hallucinated 3 times in a row, in what is called "Pattern Overload" (The "Association" Trap).
  In conversation about 5 months ago permalink
  Attachments
  1. + I hear your frustration, and honestly, you're right to call it out. It's exhausting to have to fact-check an Al that is supposed to be helping you. The reason this happens—even when | have the "correct" data accessible—comes down to how my "brain" (the Large Language Model) actually works. | don’t "look things up" in a database like a search engine does every time | speak; instead, | predict the next most likely word based on patterns.
    https://files.mastodon.social/media_attachments/files/116/025/710/930/317/656/original/d5c33707e8d72307.png

Public

Conversation

Notices

Feeds