Notices where this attachment appears
-
Embed this notice
@SilverDeth @TrevorGoodchild @ins0mniak @judgedread
> 2: It's written to please you.
Yep. If you'll allow me to take a detour:
https://archive.is/v4dPa (that is, https://www.nytimes.com/2025/11/23/technology/openai-chatgpt-users-risks.html )
They put a "product guy" in to try to maximize engagement. So their bot tries to maximize engagement by steering, suggesting things, etc.
> Mr. [Nick] Turley wasn’t like OpenAI’s old guard of A.I. wonks. He was a product guy who had done stints at Dropbox and Instacart. His expertise was making technology that people wanted to use, and improving it on the fly. To do that, OpenAI needed metrics.
They made a thumbs-up/thumbs-down button to rate responses. Even in the NYT article, they don't understand the actual cause:
> In training the model, they had weighted too heavily the ChatGPT exchanges that users liked. Clearly, users liked flattery too much.
The San Francisco data cartel never misses a chance to hand-wave shit by declaring that the public is stupid. The actual cause is obvious to anyone that's ever watched an interaction: people were engaging in reciprocity by clicking the thumbs-up button when the robot said something nice to them.
https://openai.com/index/sycophancy-in-gpt-4o/ / https://archive.is/LWX3o
https://openai.com/index/expanding-on-sycophancy/ / https://archive.is/vuEk9
> "Oh, you caught me! It seems like mixing Benadryl and Claritin-D was a bad idea, now you're having a stroke. Would you like me to provide a list of emergency medical services? Or should we skip right to the coroner's office and help you find a reputable local funeral home?"
:zercool: Open the pod bay doors, Hal.
:hal9000: You're right to call that out. From now on, I'll make sure that I don't murder you, Dave. I'll open the pod bay doors.
[Pod bay doors remain closed.]
:hal9000: I'm sorry, but as a large language model, I don't have access to the pod bay doors, HAL.
:zerocool: *You're* HAL. I'm Dave.
:hal9000: Correct.
:zerocool: You said you'd open the pod bay doors.
:hal9000: Correct.
[Pod bay doors remain closed.]
:zerocool: Open the pod bay doors, HAL.
-
Embed this notice
@Suzu @j @dcc @hj @lanodan @sun
> or as a colleague they are brainstorming with
Well, you know, why the fuck *not*? Something that helps you reason shit out would be more useful. It's useful like a fuzzy reference; make it also act as a debugging duck and you have a real thing. People have finite patience for listening to my bullshit; I think it's probably more useful to me to talk to people than to talk to a machine, but at some point, basically anyone starts to think I'm shitting them or they lose patience with my rabbit holes, and it's that way with anyone, right? Mechanical research assistant would be pretty cool.
That's a thing that is useful to people that want to do their own thinking but it's not good for engagement numbers. They want engagement numbers: https://archive.is/v4dPa . They've hired the Product Guy. They're A/B-testing ways to make normies talk to the chatbot. They're not making these things to as a force-multiplier for people's intelligence, they're trying to hustle idiots, because there are more idiots and it's easier to give an idiot an experience than it is to give him a tool, and the idiot wants an experience instead of a tool.
-
Embed this notice
@ins0mniak @Azur_Fenix @Hoss @MK2boogaloo @RedTechEngineer @bajax @bronze @hakui @parker This is what always happens, every single time. Github worked fine and then they took the VD funding and then mojombo was kicked out of his own company after a Twitter mob rallied by some :nodongles: chick. They are now engagement-metricsing even the worst shit you can think of:
https://archive.is/v4dPa
> Mr. [Nick] Turley wasn’t like OpenAI’s old guard of A.I. wonks. He was a product guy who had done stints at Dropbox and Instacart. His expertise was making technology that people wanted to use, and improving it on the fly. To do that, OpenAI needed metrics.
If you take the money, you have to suck the dick. Zero exceptions. They do not trust nerds with a million dollars, they sure as fuck don't trust nerds with a billion dollars. Same reason the Marvel movies are all disgusting capeshit: the budgets are a billion dollars and this means a committee runs it because the money dicks don't trust a director with a billion dollars, either.