Conversation

Notices

1. Embed this notice
   [realhackhistory@home]# (realhackhistory@chaos.social)'s status on Wednesday, 17-Jan-2024 00:49:52 JST [realhackhistory@home]#

   Twitter sure is fun lately

   • Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Wednesday, 17-Jan-2024 00:50:39 JST Rich Felker

     in reply to

     • Christopher Snowhill

     @chris @realhackhistory It's all encoding a hubris where the ppl working on this garbage think it's the future of how machines interact rather than a stupid party trick.

     Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Wednesday, 17-Jan-2024 00:50:40 JST Rich Felker

     in reply to

     • Christopher Snowhill

     @chris @realhackhistory Well in this case they could like pattern match for error responses like that, but the root problem is that the public interfaces to models themselves all use in-band signaling (these inane first person "I can't do that" bs responses) rather than error codes in the API layer you interface over.

   • Embed this notice
     Christopher Snowhill (chris@social.losno.co)'s status on Wednesday, 17-Jan-2024 00:50:42 JST Christopher Snowhill

     in reply to

     • Rich Felker

     @dalias @realhackhistory Gods damned, how daft do you have to be to pipe the model directly into your feed without even reading the output? Sure shows how much they care about quantity over quality of their posts. Maybe they even get paid per post, regardless of quality?

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Wednesday, 17-Jan-2024 00:50:43 JST Rich Felker

     in reply to

     @realhackhistory I love how utterly dumb everyone in LLM land is about in-band vs out-of-band signaling.

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Wednesday, 17-Jan-2024 00:51:51 JST Rich Felker

     in reply to

     • Henri
     • Christopher Snowhill

     @slyecho @chris @realhackhistory They have made very intentional, harmful interface choice to make it look like you're talking & interacting with an intelligent being rather than making structured API calls to generate text. It's harmful in both a social sense and an engineering sense.

     Haelwenn /элвэн/ :triskell: likes this.
   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Wednesday, 17-Jan-2024 00:51:53 JST Rich Felker

     in reply to

     • Henri
     • Christopher Snowhill

     @slyecho @chris @realhackhistory Yes. This thread is an indictment of the interface choice of the people pushing this stuff, not anything inherent.

   • Embed this notice
     Henri (slyecho@mdon.ee)'s status on Wednesday, 17-Jan-2024 00:51:55 JST Henri

     in reply to

     • Rich Felker
     • Christopher Snowhill

     @dalias @chris @realhackhistory If you have the possibility to run your own algorithm to select the next token you can do much more powerful things, like constrain it to a specified grammar, etc. but OpenAI does not offer that interface to users.

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Wednesday, 17-Jan-2024 00:51:56 JST Rich Felker

     in reply to

     • Henri
     • Christopher Snowhill

     @slyecho @chris @realhackhistory I'm really talking about a broader pattern of lacking OOB signaling. The whole "everything is a prompt" is very much the same type of mistake as "everything is a query string" in SQL, vs queries being structured. Except unlike SQL there isn't even any 100% reliable way to escape things with chatbot prompts.

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Wednesday, 17-Jan-2024 00:51:58 JST Rich Felker

     in reply to

     • Henri
     • Christopher Snowhill

     @slyecho @chris @realhackhistory My understanding is that it's a mix of model training and hacks outside the actual model that attempt to prevent these bots from spitting out "inappropriate" responses. But...

   • Embed this notice
     Henri (slyecho@mdon.ee)'s status on Wednesday, 17-Jan-2024 00:52:01 JST Henri

     in reply to

     • Rich Felker
     • Christopher Snowhill

     @dalias @chris @realhackhistory There are no error codes, the model is trained to respond like that, or in a similar wording because it's a language model after all.

     There are actually out-of-band signalling tokens that are not mapped to Unicode input or output, but the model can also ignore them just like a typo in the user's text.