Public
- Public
- Network
- Groups
- Featured
- Popular
- People

Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents: [begin response from Opus] Here is the most relevant sentence in the documents: "The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association." However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings. [end response from Opus]

Download link

Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents: [begin response from Opus] Here is the most relevant sentence in the documents: "The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association." However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings. [end response from Opus]
https://files.mastodon.social/media_attachments/files/112/048/544/551/916/471/original/0fd607d556d1e6b2.png

Notices where this attachment appears

Embed this notice
FeralRobots (feralrobots@mastodon.social)'s status on Wednesday, 06-Mar-2024 23:19:49 JST FeralRobots
in reply to

Put another way: Alex is basically telling Claude 3 ("Opus") that he's running a test on it, & is excited when Claude (a system for analyzing & producing human-plausible representations of similar text) "recognizes" a needle-testing prompt and produces text that's plausibly consistent with needle-testing.
What one SHOULD then do is remake one's tests (or better-sandbox the model). Instead, Alex leaps to concluding the model is self-aware.
https://twitter.com/alexalbert__/status/1764722513014329620

In conversation about a year ago from mastodon.social permalink