Conversation

Notices

1. Embed this notice
   Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Saturday, 10-Aug-2024 14:25:07 JST Emi Yusa

   what goddamn shitty mirrors does python use
   i get 10% of my internet speed no matter which network i am on

   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:25:06 JST iced depresso

     in reply to
     @GNUxeava well see the mirrors are also written in python, so its just slow :blobcatgooglybadumtss:
   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:29:06 JST iced depresso

     in reply to
     @GNUxeava
     > when the lead programmer is writing a file transfer system
     :neocat_floof_mug:
   • Embed this notice
     Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Saturday, 10-Aug-2024 14:29:08 JST Emi Yusa

     in reply to

     • iced depresso

     @icedquinn@blob.cat to top it off, pip cannot resume downloads so it is downloading torch for the fifth time

   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:32:58 JST iced depresso

     in reply to
     @GNUxeava python was somewhat nicer than writing everything in c99.
   • Embed this notice
     Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Saturday, 10-Aug-2024 14:32:59 JST Emi Yusa

     in reply to

     • iced depresso

     @icedquinn@blob.cat everything around python is just so cursed and the world decided to move with it of all things
     well makes sense cuz the world is just as cursed
     shitty people attract shitty stuff and do shitty stuff
     why am i here

   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:34:40 JST iced depresso

     in reply to
     @GNUxeava idk blob i knew when i saw someone say they were working on a file storage endpoint that they were scuffed.
     
     copying files around is such a fucking trash fire of "it seems like it should be so simple. so anyway that's why we had to add probablistic polynomial hashes to split up the files :neocat_floof_mug:"
   • Embed this notice
     Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Saturday, 10-Aug-2024 14:34:41 JST Emi Yusa

     in reply to

     • iced depresso

     @icedquinn@blob.cat the ecosystem

   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:38:37 JST iced depresso

     in reply to
     @GNUxeava well rsync IS doing polynomial hashes. :neocat_bottom: that's.. how it does its incremental transfers of large files.
   • Embed this notice
     Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Saturday, 10-Aug-2024 14:38:38 JST Emi Yusa

     in reply to

     • iced depresso

     @icedquinn@blob.cat probablistic polynomial hashes to split up the filesdeath by x-ray
     use the normal fucking rsync or just http like the rest of the internet

   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:41:55 JST iced depresso

     in reply to

     • iced depresso
     @GNUxeava http is a bit :neocat_googly: at file transfers. you can put in a content-length (or not? file could be "i'mma shit over the pipe until i close the link" length), and there isn't really a "did you recieve that correctly?" checksum, there isn't really a way to say "i still have 56% of that file before we shit the bed, can i have the rest?"
     
     the closest is a range request, which is SUPPOSED to be supported by standards compliant servers but is it? :blobcatdunno:
     
     rsync on the other hand rolling hashes files in to chunks, and communicates to itself (the other server also has rsync...) about what blocks are present and if they transferred OK, stuff like that. it's a pain. most people just shit the file over the pipe and assume nothing will ever go wrong. for sufficiently small data... it doesn't? or it does and you just retry.
     
     when you have 2 gigabyte artifacts like torch .. :neofox_up_paws:
   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:43:29 JST iced depresso

     in reply to

     • iced depresso
     @GNUxeava if you ever get curious this is talked about by the casync docs (possibly poetterings only good program) and how they do some trickery with chunking and range requests to manage large file transfers in a resumable/deduplicating way.
     
     basically file transfer is so simple in concept everyone just goes and does it wrong but its actually a pain in the boobs.
   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:46:13 JST iced depresso

     in reply to
     @GNUxeava :blobcatthonkang: i would be surprised if pip doesn't depend on Requests, actually. that's a very ubiquitous library and i think it does support those things if you ask it to.
   • Embed this notice
     Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Saturday, 10-Aug-2024 14:46:14 JST Emi Yusa

     in reply to

     • iced depresso

     @icedquinn@blob.cat yeah
     it is a miracle that curl and wget and aria2 can handle things so nicely
     so why doesn't pip use those libraries?

   • Embed this notice
     iced depresso (icedquinn@blob.cat)'s status on Saturday, 10-Aug-2024 14:58:13 JST iced depresso

     in reply to
     @GNUxeava yeah.
   • Embed this notice
     Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Saturday, 10-Aug-2024 14:58:14 JST Emi Yusa

     in reply to

     • iced depresso

     @icedquinn@blob.cat

   • Embed this notice
     Emi Yusa (gnuxeava@mk.absturztau.be)'s status on Saturday, 10-Aug-2024 14:58:15 JST Emi Yusa

     in reply to

     • iced depresso

     @icedquinn@blob.cat most competent python dev
     i am not going to manually hunt hundreds of dependencies with curl and then do a pip install