Conversation

Notices

1. Embed this notice
   Sophie Schmieg (sophieschmieg@infosec.exchange)'s status on Saturday, 04-Jan-2025 07:38:04 JST Sophie Schmieg

   I was forwarded this screenshot and it just is living rent free in my head right now.

   • MortSinyx likes this.
   • Rich Felker repeated this.
   • Embed this notice
     Sophie Schmieg (sophieschmieg@infosec.exchange)'s status on Saturday, 04-Jan-2025 12:00:08 JST Sophie Schmieg

     in reply to

     In case you do not know how GenAI works, here is a very abridged description:
     First you train your model on some inputs. This is using some very fancy linear algebra, but can be seen as mostly being a regression of some sorts, i.e. a lower dimensional approximation of the input data.
     Once training is completed, you have your model predict the next token of your output. It will do so by creating a list of possible tokens, together with a rank of how good of a fit the model considers the specific token to be. You then randomly select from that list of tokens, with a bias to higher ranked tokens. How much bias your random choice has depends on the "temperature" parameter, with a higher temperature corresponding to a less biased, i.e. more random selection.

     Now obviously, this process consumes a lot of randomness, and the randomness does not need to be cryptographically secure, so you usually use a statistical random number generator like the Mersenne twister at this step.

     So when they write "using a Gen AI model to produce 'true' random numbers", what they're actually doing is using a cryptographically insecure random number generator and applying a bias to the random numbers generated, making it even less secure. It's amazing that someone can trick anyone into investing into that shit.

     Haelwenn /элвэн/ :triskell: likes this.
     Rich Felker repeated this.
   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Saturday, 04-Jan-2025 12:02:26 JST Rich Felker

     in reply to

     @sophieschmieg LMAO what??? There are ppl trying to use LLM output as RNG??? And thinking "I'm too stupid to understand how it works so that means it's secure!!!111" ??? 🤦

     🤏 🎻 when they get pwned. I'm out of patience for the LLM fan 🤡 🚗

     Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Saturday, 04-Jan-2025 12:05:44 JST Rich Felker

     in reply to

     @sophieschmieg BTW not criticizing your choice of MT as an illustration because it's exactly the sort of thing these bozos would know by name, but it's utterly the worst choice of deterministic PRNG. Gratuitously large state, poor output quality. Even a 128bit or possibly even 64bit LCG throwing away lower bits is better.

   • Embed this notice
     niconiconi (niconiconi@mk.absturztau.be)'s status on Saturday, 04-Jan-2025 12:06:15 JST niconiconi

     in reply to

     @sophieschmieg@infosec.exchange Ironically most GenAI implementations have troubles on producing deterministic output due to floating point errors, inconsistent batching, etc. Not random enough for crypto, but random enough to create replication problems. It's what I call Murphy's Duality Law - In engineering, when a system can show both the property "A" and its negation "not A" depending on the specific context, it's always the opposite of what your application needs.

     Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Cassander (drsbaitso@infosec.exchange)'s status on Saturday, 04-Jan-2025 12:06:30 JST Cassander

     in reply to

     @sophieschmieg If LLMs are snake oil, this "AI RNG" is meta-snake oil. It's like expecting a homeopathy distillation of horse dewormer will cure Covid.

     It's so obviously fake that I can't even find a good metaphor to explain how bad it is.

     Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Dan Sugalski (wordshaper@weatherishappening.network)'s status on Saturday, 04-Jan-2025 12:07:51 JST Dan Sugalski

     in reply to

     @sophieschmieg "We trained our neural net on thousands of images of lava lamps and can now generate an endless series of random numbers!"

     Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Dan Sugalski (wordshaper@weatherishappening.network)'s status on Saturday, 04-Jan-2025 12:07:52 JST Dan Sugalski

     in reply to

     @sophieschmieg https://giphy.com/explore/facepalm

     Rich Felker repeated this.
   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Saturday, 04-Jan-2025 12:08:50 JST Rich Felker

     in reply to

     • Teknikal_Domain
     • Anthropy :verified_dragon:

     @anthropy @tek_dmn @sneexy That's why we call em toots on Mastodon.

   • Embed this notice
     Anthropy :verified_dragon: (anthropy@mastodon.derg.nz)'s status on Saturday, 04-Jan-2025 12:08:52 JST Anthropy :verified_dragon:

     in reply to

     • Teknikal_Domain

     @tek_dmn @sneexy I think I might've shitposted my pants :pensive_blob:

     Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Anthropy :verified_dragon: (anthropy@mastodon.derg.nz)'s status on Saturday, 04-Jan-2025 12:08:54 JST Anthropy :verified_dragon:

     in reply to

     • Teknikal_Domain

     @tek_dmn @sneexy oh yea well HNNNGG *shitposts super hard like omg why would you shitpost that hard that's dangerous and bad for your blood pressure*

   • Embed this notice
     Teknikal_Domain (tek_dmn@mastodon.tekdmn.me)'s status on Saturday, 04-Jan-2025 12:08:55 JST Teknikal_Domain

     in reply to

     • Anthropy :verified_dragon:

     @anthropy Nah, you're not.

     The only true random shitpost generator on here is @sneexy

     @sophieschmieg

   • Embed this notice
     Anthropy :verified_dragon: (anthropy@mastodon.derg.nz)'s status on Saturday, 04-Jan-2025 12:08:56 JST Anthropy :verified_dragon:

     in reply to

     @sophieschmieg I know you have your methods, but if interested I've become the first only true random shitpost generator using my brain trained on naturally recurring shitposts with enough anthropy and randomness to defend against a whole quanta of warm tea

   • Embed this notice
     Mx Amber Alex (she/it) (amberage@eldritch.cafe)'s status on Saturday, 04-Jan-2025 12:11:54 JST Mx Amber Alex (she/it)

     in reply to

     @sophieschmieg who would win:

     • random numbers from radioactive decay
     • heating the oceans and consuming the energy output of a small country to produce a bad imitation of radioactive decay
     Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Mx Amber Alex (she/it) (amberage@eldritch.cafe)'s status on Saturday, 04-Jan-2025 12:11:55 JST Mx Amber Alex (she/it)

     in reply to

     @sophieschmieg ??? help

     Rich Felker repeated this.
   • Embed this notice
     Paul Hoffman (paulehoffman@infosec.exchange)'s status on Saturday, 04-Jan-2025 12:13:23 JST Paul Hoffman

     in reply to

     @sophieschmieg Thank you for sharing the rent-free.

     And, yes, this person is real, and apparently believes he is doing something interesting. https://www.linkedin.com/in/eric-dresdale/ (don't have liquids in your mouth when reading…).

     Haelwenn /элвэн/ :triskell: likes this.
     Rich Felker repeated this.
   • Embed this notice
     Sophie Schmieg (sophieschmieg@infosec.exchange)'s status on Saturday, 04-Jan-2025 12:13:27 JST Sophie Schmieg

     in reply to

     • Paul Hoffman

     @paulehoffman oh yeah, he founded a company, apparently. It doesn't say if he actually found anyone dumb enough to invest in it, but yeah.

     Remember, it's only fraud if you actually understand that what you're doing is nonsense!

   • Embed this notice
     Valerie Aurora (vaurora@wandering.shop)'s status on Saturday, 04-Jan-2025 19:59:13 JST Valerie Aurora

     in reply to

     • Gus

     @sophieschmieg @projectgus makes me yearn for the days of programmers merely believing that SHA-1 would never be broken