Conversation

Notices

1. Embed this notice
   iced depresso (icedquinn@blob.cat)'s status on Sunday, 27-Apr-2025 18:21:20 JST iced depresso
   going to be extremely funny if AI was solved in the 90s and the answer ends up being really inefficient versions of the mixture models we already had.
   
   which is already partly true looking, as KANs are .. kinda related to what gaussian mixtures actually did, in the sense of 'learning' curves in to the mixture nodes and re-constructing functions that way.
   
   there's even a paper from years ago where you can just use gradient descent on mixture models and it works better than the old EM algorithm did.
   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Sunday, 27-Apr-2025 18:29:16 JST iced depresso

     in reply to

     • meeper
     @meeper no?
   • Embed this notice
     meeper (meeper@shitposter.world)'s status on Sunday, 27-Apr-2025 18:29:17 JST meeper

     in reply to
     @icedquinn don't KAN's have kinda dissapointing performance?
   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Sunday, 27-Apr-2025 18:44:54 JST iced depresso

     in reply to

     • meeper
     @meeper afaik there hasn't been a whole lot of proper testing done on them.
     
     i came across them while reading about liquid state machines and spike net stuff doing very impressive and near-brain things, but i haven't tried to get them to compute things yet. just been curious because the way they are built seems like they might get around some issues with contextual memory (because splines) and they might do well at wide-but-shallow nets since the derivatives don't actually mix all over everything like they do in other layouts
   • Embed this notice
     meeper (meeper@shitposter.world)'s status on Sunday, 27-Apr-2025 18:44:55 JST meeper

     in reply to
     @icedquinn
     I've not really looked in depth into them, but the opinions I've been seeing is that they were somewhat overhyped and the gains were not too significant combined with greater difficulty in training