Notices by Arvind Narayanan (randomwalker@mastodon.social)

OpenAI mitigates ChatGPT’s biases using fine tuning and reinforcement learning. These methods affect only the model’s output, not its implicit biases (the stereotyped correlations that it's learned). Since implicit biases can manifest in countless ways, OpenAI is left playing whack-a-mole, reacting to examples posted on social media.

People have been posting glaring examples of ChatGPT’s gender bias, like arguing that attorneys can't be pregnant. So @sayashk and I tested ChatGPT on WinoBias, a standard gender bias benchmark. Both GPT-3.5 and GPT-4 are about 3 times as likely to answer incorrectly if the correct answer defies gender stereotypes — despite the benchmark dataset likely being included in the training data. https://aisnakeoil.substack.com/p/quantifying-chatgpts-gender-bias

New blog: Twitter released some of its code, but not the ML models that recommend tweets, while cutting off researchers' API access. It shows the limited utility of source code transparency.

The one useful thing we learned was how Twitter defines engagement. But that wasn't actually part of the source and was published separately! This type of transparency about how algorithms are configured should be considered essential, and doesn't require release of source code.

https://knightcolumbia.org/blog/twitter-showed-us-its-algorithm-what-does-it-tell-us

The AI moratorium letter only fuels AI hype. It repeatedly presents speculative, futuristic risks, ignoring the version of the problems that are already harming people. It distracts from the real issues and makes it harder to address them. The letter has a containment mindset analogous to nuclear risk, but that’s a poor fit for AI. It plays right into the hands of the companies it seeks to regulate. By @sayashk and me. https://aisnakeoil.substack.com/p/a-misleading-open-letter-about-sci

I recently came across a fascinating theory that traces the crisis of American democracy to the fact that everyday people used to participate in local governance, but they don't anymore (I forget why... increasing urbanization? The nationalization of politics?) The distance from governance leads to distrust in government (and institutions in general).

Unfortunately I can't remember the source. If anyone can point me to it, I'd be eternally grateful.

The philosopher Harry Frankfurt defined bullshit as speech intended to persuade without regard for the truth. By this measure, OpenAI’s new chatbot ChatGPT is the greatest bullshitter ever. Large Language Models (LLMs) are trained to produce plausible text, not true statements. So using ChatGPT in its current form would be a bad idea for applications like education or answering health questions.

Despite this, there are three areas where LLMs can be extremely useful: https://aisnakeoil.substack.com/p/chatgpt-is-a-bullshit-generator-but

Here are three kinds of tasks where @sayashk and I think ChatGPT can shine, despite its inability to discern truth in general:

1. Tasks where it’s easy for the user to check if the bot’s answer is correct, such as debugging help.

2. Tasks where truth is irrelevant, such as writing fiction.

3. Tasks for which there does in fact exist a subset of the training data that acts as a source of truth, such as language translation.

https://aisnakeoil.substack.com/p/chatgpt-is-a-bullshit-generator-but

Algorithms aren't the enemy. Chronological feeds don't scale and the signal-to-noise ratio will plummet if this ever gets popular. The real problems with today's algorithmic feeds are non-transparency, lack of choice, and optimizing for engagement instead of healthy discourse.

Open-source is a perfect opportunity to fix all this. Have there been any efforts to create a Mastodon instance with a (community governed) ranking algorithm? Is that technically feasible? Or is the idea simply anathema?