Conversation

Notices

1. Embed this notice
   iced depresso (icedquinn@blob.cat)'s status on Sunday, 18-Feb-2024 16:44:55 JST iced depresso
   > up to 80% of a large neural network is doing more or less nothing
   
   :blobcatgoogly: science
   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Sunday, 18-Feb-2024 16:48:20 JST iced depresso

     in reply to

     • 🅴🅻🅸🆇🆇 :verified:
     @elixx :neocat_googly:
   • Embed this notice
     🅴🅻🅸🆇🆇 :verified: (elixx@social.drastical.tech)'s status on Sunday, 18-Feb-2024 16:48:24 JST 🅴🅻🅸🆇🆇 :verified:

     in reply to

     @icedquinn
     needs hyperthreading

   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Sunday, 18-Feb-2024 16:49:59 JST iced depresso

     in reply to

     • Lizzie
     @lizzie i think the brain is sparse connection sparse layer. most networks trained are dense for some reason.
     
     numenta has a paper about 'complimentary sparsity' and how making both elements sparse is letting them cut the majority of processing out of the models.
     
     i'm reading the top-kast paper rn that talks about how they were able to shimmy sparse layers in to pytorch
   • Embed this notice
     Lizzie (lizzie@brain.worm.pink)'s status on Sunday, 18-Feb-2024 16:50:05 JST Lizzie

     in reply to

     @icedquinn makes sense tbh bc actual neuron training produces similar results but brains can prune