Conversation

Notices

1. Embed this notice
   Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:15:17 JST Sexy Moon
   I need to trade in my Mac for a new one with more RAM :mexican_sadge:
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 06-Feb-2024 23:16:46 JST 翠星石

     in reply to
     @Moon For your freedoms sake, why not trade in your mac for a GNUbooted thinkpad?
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:18:21 JST Sexy Moon

     in reply to

     • Rusty Crab
     @RustyCrab You can run GIANT models on Macs since they have unified system and vram. I'll be getting a lot put in it if I do this.
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: likes this.
   • Embed this notice
     Rusty Crab (rustycrab@clubcyberia.co)'s status on Tuesday, 06-Feb-2024 23:18:22 JST Rusty Crab

     in reply to
     @Moon please to be paying $3,000 for 24 gigs of ram (don't worry it's magic ram that holds more ram than regular ram)
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:20:20 JST Sexy Moon

     in reply to

     • 翠星石
     @Suiseiseki I have a thinkpad, I need a system to do AI with
   • Embed this notice
     FED Enjoyer (red_hat@nicecrew.digital)'s status on Tuesday, 06-Feb-2024 23:20:51 JST FED Enjoyer

     in reply to

     • Rusty Crab
     We soldered ram on to your ram so you could ra while you ra.
     Sexy Moon likes this.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:22:26 JST Sexy Moon

     in reply to

     • taylor
     @nik :sadge:
   • Embed this notice
     taylor (nik@misskey.bubbletea.dev)'s status on Tuesday, 06-Feb-2024 23:22:27 JST taylor

     in reply to

     @Moon@shitposter.club why not just open it up and upgrade the ram sticks :^)

   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 06-Feb-2024 23:23:53 JST 翠星石

     in reply to
     @Moon Have you considered not running proprietary malware bullshit generators?
   • Embed this notice
     taylor (nik@misskey.bubbletea.dev)'s status on Tuesday, 06-Feb-2024 23:25:19 JST taylor

     in reply to

     @Moon@shitposter.club this is why I'll never purchase a Mac tbh not worth paying $2500 for something that will need $1500 in maintenance cost if anything ever happens to it
     
     my framework has its own problems but I can take it apart in a couple minutes, was helpful when I dunked it in water a few weeks ago

     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: likes this.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:25:31 JST Sexy Moon

     in reply to

     • 翠星石
     @Suiseiseki I only run libre AI software and models.
   • Embed this notice
     why? (why@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:26:45 JST why?

     in reply to
     @Moon or rackmount mac pro with 192gb 🤤
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:26:45 JST Sexy Moon

     in reply to

     • why?
     @why I kind of wish I would have done this instead of build the other desktop pc
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:28:09 JST Sexy Moon

     in reply to

     • why?
     @why holy shit that is expensive. also why does the rack mount only do 192 when you can get a macbook pro with 128.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:29:41 JST Sexy Moon

     in reply to

     • cool_boy_mew
     • why?
     @coolboymew @why Surprised there's not a Saudi prince option for 1TB of RAM.
   • Embed this notice
     cool_boy_mew (coolboymew@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:29:42 JST cool_boy_mew

     in reply to

     • why?
     @Moon @why because Apple
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:34:18 JST Sexy Moon

     in reply to

     • on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ
     @lain the what
   • Embed this notice
     on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ (lain@lain.com)'s status on Tuesday, 06-Feb-2024 23:34:20 JST on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ

     in reply to
     @Moon or get the mac studio
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:35:26 JST Sexy Moon

     in reply to

     • on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ
     @lain woah I never saw such a thing.
   • Embed this notice
     on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ (lain@lain.com)'s status on Tuesday, 06-Feb-2024 23:35:28 JST on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ

     in reply to
     @Moon https://www.apple.com/mac-studio/
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 06-Feb-2024 23:36:01 JST 翠星石

     in reply to
     @Moon I've looked for such, but everything I've looked at as been proprietary.
     
     Is the software really free if it can't be run at all on freedom respecting computers?
     Sexy Moon likes this.
   • Embed this notice
     on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ (lain@lain.com)'s status on Tuesday, 06-Feb-2024 23:36:57 JST on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ

     in reply to
     @Moon it seems to be pretty popular with the LLM crowd because it can run even the largest models
   • Embed this notice
     Piggo :verified_horse: (piggo@piggo.space)'s status on Tuesday, 06-Feb-2024 23:38:28 JST Piggo :verified_horse:

     in reply to

     • on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ
     @Moon @lain lmao over half of the thing is just a massive cooler
     Sexy Moon likes this.
     Pleroma-tan repeated this.
   • Embed this notice
     on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ (lain@lain.com)'s status on Tuesday, 06-Feb-2024 23:39:36 JST on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ

     in reply to

     • Piggo :verified_horse:
     @piggo @Moon not everything can be as elegant and tasteful as a pc ML setup.
     feld likes this.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:40:56 JST Sexy Moon

     in reply to

     • 翠星石
     @Suiseiseki There are various model files in different sizes. They need to fit inside your VRAM. So, you can get normal-sized models and run. them on Linux. I do in fact have a Linux box with 24GB of VRAM but that is still not enough for the largest models. A Mac, if you spend the money, can run a model in over 190GB.
     feld and :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: like this.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 06-Feb-2024 23:41:25 JST 翠星石

     in reply to

     • 翠星石
     @Moon If the software is actually free and RAM usage is the blocker, I would suggest getting a KGPE-D16 and installing 128 GB of DDR3 ECC RAM into it (or 256GB if you want).
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:42:17 JST Sexy Moon

     in reply to

     • 翠星石
     @Suiseiseki It has to be VRAM or system-unified RAM. There may be a free system with unified RAM but I have not seen one.
   • Embed this notice
     on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ (lain@lain.com)'s status on Tuesday, 06-Feb-2024 23:43:08 JST on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ

     in reply to

     • 翠星石
     @Suiseiseki @Moon there are like 5 different apache 2.0 LLMs and tons of engines to run them
     Sexy Moon likes this.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 06-Feb-2024 23:43:51 JST 翠星石

     in reply to
     @Moon The software isn't free if it can only be compiled against proprietary libraries and only run on certain models of proprietary GPUs sorry.
     
     I don't see how Linux is relevant in this context, the proprietary GPU libraries manage the VRAM really.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:45:19 JST Sexy Moon

     in reply to

     • 翠星石
     @Suiseiseki Oh that is true, on my Linux machine it has to use the proprietary nvidia driver. no way around that.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 06-Feb-2024 23:45:33 JST 翠星石

     in reply to

     • on-lain ✔ᵛᵉʳᶦᶠᶦᵉᵈ
     @lain Apache 2.0 doesn't do much to ensure the software is free and remains free, so I often need to check if certain packages licensed under the Apache 2.0 are actually free software.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Tuesday, 06-Feb-2024 23:47:19 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     @Suiseiseki @Moon you probably could, given enough time and effort, run the models on asahi using free reverse engineered gpu/tpu drivers because apple doesn't sabotage re efforts for it. no such comfort awaits the nvidia user.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:49:30 JST Sexy Moon

     in reply to

     • Rusty Crab
     • Johnny Peligro
     @mischievoustomato @RustyCrab It's better but it still doesn't justify the price. They are just an upscale brand and they price commensurately.
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: likes this.
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Tuesday, 06-Feb-2024 23:49:31 JST Johnny Peligro

     in reply to

     • Rusty Crab
     it's due to a number of factors that macOS uses less ram more or less, but a common thing in OSs is memory compression.
     That said, arm macbook's memory bandwidth, and i suppose also speed, is pretty crazeh
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Tuesday, 06-Feb-2024 23:51:49 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @Moon the highest end alchemist card currently available is still pretty weak for ai stuff afaik, whether battlemage changes the situation remains to be seen
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Tuesday, 06-Feb-2024 23:51:50 JST Johnny Peligro

     in reply to

     • 翠星石
     no intel oneapi support yet? :(
   • Embed this notice
     vic (vic@seal.cafe)'s status on Tuesday, 06-Feb-2024 23:51:54 JST vic

     in reply to

     • 翠星石
     @Moon @Suiseiseki > if you spend the money
     
     wowzers
     Sexy Moon likes this.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:52:09 JST Sexy Moon

     in reply to

     • 翠星石
     • vic
     @vic @Suiseiseki Yeah, I cannot justify that price for only 192GB of RAM.
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Tuesday, 06-Feb-2024 23:52:16 JST Johnny Peligro

     in reply to

     • 翠星石
     system unified with a gpu/npu is basically new ryzens/intel laptop chips
     Sexy Moon likes this.
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Tuesday, 06-Feb-2024 23:52:28 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     what if the models only run on cuda on linux though
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Tuesday, 06-Feb-2024 23:52:28 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @Moon then you're screwed. thankfully, less and less things are actually like this with each passing year.
   • Embed this notice
     vic (vic@seal.cafe)'s status on Tuesday, 06-Feb-2024 23:52:51 JST vic

     in reply to

     • 翠星石
     @Moon @Suiseiseki yeah but it's UNIFIED
     Sexy Moon likes this.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Tuesday, 06-Feb-2024 23:54:25 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @Moon from what I've seen most everything calls out to a few big libraries like pytorch and if those are supported on your hardware you should be fine.
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Tuesday, 06-Feb-2024 23:54:26 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     how is it >now< though? from what I've seen everything either supports cuda, or rocm, but rocm support is a gamble from amd
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:54:27 JST Sexy Moon

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @allison yeah I think you gotta go proprietary.
     
     and nvidia drivers are such garbage. they break every time I upgrade and I have to recompile everything.
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: likes this.
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Tuesday, 06-Feb-2024 23:54:28 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     i guess even meteor lake npu/igpus are too weak for that too :(
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Tuesday, 06-Feb-2024 23:56:21 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @Moon @Suiseiseki @mischievoustomato it absolutely can not be understated how much apple hardware just works for this usecase in comparison.
     feld likes this.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 06-Feb-2024 23:56:33 JST 翠星石

     in reply to

     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     @allison >run the models on asahi using free reverse engineered gpu/tpu drivers because apple doesn't sabotage re efforts for it
     I wouldn't call those drivers free, considering that Aarch64 Macs have a bootloader that loads a huge amount of proprietary software like peripheral software onto the GPU etc at boottime (proprietary software lovers really love that "feature") and it's not possible cryptographically to free that software.
     
     While apple for now doesn't dedicate all their effort to stop half of the GPU/TPU driver from being re-written, half of the driver being free doesn't really result it as a whole being free.
     
     >no such comfort awaits the nvidia user.
     Actually it kind of does 700 series and below with the free nouveau driver.
     
     OpenCL support is currently pretty poor, but those GPUs do current have a 95% free driver considering that the Linux driver and the peripheral software are free - the remaining issue is that VBIOS's stored in an EEPROM chip on the GPUs are proprietary - but freeing such VBIOS's seems possible with enough effort, as 700 series and below lack cryptographic handcuffs.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:57:15 JST Sexy Moon

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     • Johnny Peligro
     @allison @Suiseiseki @mischievoustomato I git cloned llama.cpp today and ran make -j and I had a working system in less than five minutes. It took over a day to get it working on my Linux box.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:58:03 JST Sexy Moon

     in reply to

     • 翠星石
     • vic
     @vic @Suiseiseki you can run a ton of stuff on 24GB.
   • Embed this notice
     vic (vic@seal.cafe)'s status on Tuesday, 06-Feb-2024 23:58:09 JST vic

     in reply to

     • 翠星石
     @Moon @Suiseiseki So I really can't just go get a MEGA ULTRA GAMER ELITE PC with a 4090 (24 GB VRAM) and run the biggest AI stuff on it by having it share/offload some of the model to system RAM? I've never really delved into the big AI/GPU stuff.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Tuesday, 06-Feb-2024 23:58:23 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @Moon @Suiseiseki @mischievoustomato what are the vram requirements for llama.cpp? seems like something I might want to mess with on my intel/intel rig lol
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Tuesday, 06-Feb-2024 23:59:05 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     a lot of people went crazy over apple silicon, and also apple released libraries and shit iirc
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Tuesday, 06-Feb-2024 23:59:05 JST 翠星石

     in reply to

     • Johnny Peligro
     @mischievoustomato Of course sheep would go crazy over a new apple product, even if that silicon isn't even apple's.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Tuesday, 06-Feb-2024 23:59:51 JST Sexy Moon

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     • Johnny Peligro
     @allison @Suiseiseki @mischievoustomato it all depends on the model you run. 16GB will run a lot of stuff. 12GB is will too but it's not big enough for some useful models.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:00:03 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @Moon Still can't believe the spaghetti I spilled when I told him I preferred Motorola/PowerPC in our one IRL encounter lmfao
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:00:04 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     fucking zased rig, blessed by pat gelsinger
     Sexy Moon likes this.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:00:37 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @Moon @Suiseiseki @mischievoustomato Yeah I have 16GB, so I assume that's the bare minimum for most useful stuff?
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Wednesday, 07-Feb-2024 00:00:38 JST Sexy Moon

     in reply to

     • 翠星石
     • Hoss Delgado
     @Hoss @Suiseiseki just get laid and don't worry about such things
   • Embed this notice
     Hoss Delgado (hoss@shitpost.cloud)'s status on Wednesday, 07-Feb-2024 00:00:39 JST Hoss Delgado

     in reply to

     • 翠星石
     >Is the software really free if it can't be run at all on freedom respecting computers?
     Once I reach wizard status I may be able to answer such questions.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:01:33 JST 翠星石

     in reply to

     • pomstan
     @pomstan Freedom 0, the freedom to run the software.
     
     Intentionally writing software in a way that ensures it only ever runs at all with proprietary software arguably means that the software doesn't respect freedom 0, as you can't run it in freedom after all.
   • Embed this notice
     pomstan (pomstan@xn--p1abe3d.xn--80asehdb)'s status on Wednesday, 07-Feb-2024 00:01:34 JST pomstan

     in reply to

     • 翠星石

     @Suiseiseki @Moon

     The software isn't free

     which of the 4 freedoms is violated

   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:03:35 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @Moon Even by the time I was 8 or 9 I was all in on the "damn I hate x86" train, I actually mainly used PowerPC iron at home for a good chunk of the late 2000s and early 2010s and I honestly quite enjoyed the experience
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:03:36 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     that's interesting, why lmao?
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:05:46 JST 翠星石

     in reply to

     • Johnny Peligro
     @mischievoustomato I don't see anything revolutionary about getting existing CPU and GPU soft cores, making a few changes and slapping them all into an Aarch64 SoC and paying TSMC ridiculous sums to fab them at the smallest node size.
     
     Sure, a big improvement in processing power/power consumption resulted, but that's hardly revolutionary.
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:05:48 JST Johnny Peligro

     in reply to

     • 翠星石
     it was pretty revolutionary. a lot of people that weren't even apple users moved to apple because of it
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:07:30 JST 翠星石

     in reply to

     • pomstan
     @pomstan Yes, it's entirely possible to free trapped free software by removing proprietary dependencies and replacing them with free implementations.
   • Embed this notice
     pomstan (pomstan@xn--p1abe3d.xn--80asehdb)'s status on Wednesday, 07-Feb-2024 00:07:31 JST pomstan

     in reply to

     • 翠星石

     @Suiseiseki nobody is forcing you to run software in a specific way. you can reimplement proprietary libs by yourself according to the expectations expressed in that free software

   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:09:05 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @Moon It's just a way nicer looking asm and sanely designed architecture compared to x86. I actually learned a bit of x86 asm around that point and it felt incredibly dirty to me. As for power efficiency concerns, that was down to economies of scale, Apple's partners fumbled hard with the G5 by essentially making their version of the Pentium 4 and Intel at the time had a really good contingency plan in the form of the Israeli-developed Pentium M and Core lineage. Eventually Intel repeated IBM's G5 mistake with Skylake, Apple decided to switch architectures *again*, and now they have ARM64 which if you squint really hard you could almost imagine is recent POWER.
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:09:06 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     when i was 8 or 9 i was too small to know about that, and my house didn't have internet. why did you prefer it over x86? everyone seemed to prefer x86 and apple moved to x86 for efficiency/performance reasons too, no?
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:09:07 JST 翠星石

     in reply to

     • Iska
     @iska Maybe such CPU language modules can be run in freedom, but the developers most of the time cannot help themselves but to make it proprietary.
     
     Please let me know about such language models and I'll see if I can find proprietary things.
   • Embed this notice
     Iska (iska@catposter.club)'s status on Wednesday, 07-Feb-2024 00:09:09 JST Iska

     in reply to

     • 翠星石

     @Suiseiseki@freesoftwareextremist.com @Moon@shitposter.club You can run the AI models on CPU completely libre though? :looks_inside:

   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:11:44 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     • Johnny Peligro
     @mischievoustomato @Moon @Suiseiseki (Also an interesting aside, the beginning of apple's in house chip design talent was when they bought a Dobberpuhl startup named P.A. Semi which was working on low-power PowerPC chips https://en.m.wikipedia.org/wiki/P.A._Semi )
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:13:58 JST 翠星石

     in reply to

     • pomstan
     @pomstan https://www.gnu.org/philosophy/java-trap.html
   • Embed this notice
     pomstan (pomstan@xn--p1abe3d.xn--80asehdb)'s status on Wednesday, 07-Feb-2024 00:13:59 JST pomstan

     in reply to

     • 翠星石

     @Suiseiseki @Moon

     trapped free software

     insane babble

   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:14:58 JST 翠星石

     in reply to

     • "; DROP TABLE Users;
     @TURBORETARD9000 AMD64 is already RISC - the CISC instructions get turned into RISC micro ops by the microcode.
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 00:14:59 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki needs RISC
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:15:00 JST Johnny Peligro

     in reply to

     • 翠星石
     whatever, might not faze you, but to many including me, those performance and efficiency gains are revolutionary. Fucking crazy.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:16:35 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @Moon Apple was so furious Intel couldn't deliver what they promised with Skylake that they were forced to go back to the drawing board when the mobile parts were unveiled, AMD also came out with Ryzen around that time and started regaining their footing directly because of this embarassment (it's also why Intel eventually brought Gelsinger back on board, not a single other person on earth could even have a prayer of a chance at fixing this mess)
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:16:36 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     isn't skylake regarded well, though? maybe im wrong
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:18:43 JST 翠星石

     in reply to

     • pomstan
     @pomstan Try reading the whole article start to end - I felt a million nerve connections assemble myself.
   • Embed this notice
     pomstan (pomstan@xn--p1abe3d.xn--80asehdb)'s status on Wednesday, 07-Feb-2024 00:18:45 JST pomstan

     in reply to

     • 翠星石

     @Suiseiseki @Moon i think i lost a hundred nerve cells just skimming through this article

   • Embed this notice
     Iska (iska@catposter.club)'s status on Wednesday, 07-Feb-2024 00:19:31 JST Iska

     in reply to

     • 翠星石

     @Suiseiseki@freesoftwareextremist.com @Moon@shitposter.club the bad part about AI is that all of it is written in python, and working with that language is a pain

     Sexy Moon likes this.
   • Embed this notice
     Iska (iska@catposter.club)'s status on Wednesday, 07-Feb-2024 00:21:03 JST Iska

     in reply to

     • 翠星石

     @Suiseiseki@freesoftwareextremist.com Mistral/Mixtral is Apache-licensed, Ollama is MIT licensed (There's other programs/libraries that can run it)

     ロミンちゃん likes this.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:21:40 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     • Johnny Peligro
     @TURBORETARD9000 @Suiseiseki @mischievoustomato No such thing as "sufficiently smart compilers"
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 00:21:41 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki I hope you're ready for Intel's Bulldozer moment
     I like x86 and all, but I'm looking forward to hard cores of Risc-V and how they compete
     I'm STILL a bit miffed that IA64 died before I could get my hands on it, it had flaws but it's got a good idea
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:21:42 JST Johnny Peligro

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     i wanna stay on x86 for many reasons + i like intel, but getting a MacBook isnt off the table
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:22:00 JST 翠星石

     in reply to

     • Iska
     @iska >Download Ollama
     >curl https://ollama.ai/install.sh | sh
     >The install script has no license (therefore it's proprietary) and installs proprietary nvidia drivers
     Hmmmmmmm
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Wednesday, 07-Feb-2024 00:22:08 JST Sexy Moon

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     • Johnny Peligro
     @allison @Suiseiseki @mischievoustomato I was always a PowerPC skeptic. I got a G4 Mac Mini and it just didn't perform as well as their benchmarks claimed.
     
     In the 64-bit space, you could get a top-end G5 and beat Intel. But when Apple switched their laptops to Intel their laptops got an immediate 2-4x speedup. I tried it and it was immediately apparent that the PowerPC hype was bullshit.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:22:37 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki @Moon Conversely, I want to leave x86 as soon as I can and I want wherever I end up next to be a soft landing with a pleasant low-level experience.
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:22:38 JST Johnny Peligro

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     i hope gelsinger gets to fix shit well, I don't wanna leave x86 yet
   • Embed this notice
     feld (feld@bikeshed.party)'s status on Wednesday, 07-Feb-2024 00:23:54 JST feld

     in reply to

     • 翠星石
     • vic
     @vic @Suiseiseki @Moon
     
     > by having it share/offload some of the model to system RAM?
     
     You're missing the point of how expensive of an operation it is for the CPU to access memory on the GPU. The latency is the problem, plus it needs to be copied over the PCIE bus (except in the situations with the newer CPUs and GPUs that improve upon this operation). On a unified memory platform the CPU can access the same memory address space that the GPU processor operates on. No copies, no additional latency.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:26:17 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @Moon @Suiseiseki @mischievoustomato The reasons I liked the architecture had little to do with Apple's marketing and everything to do with the architecture itself. When it comes to implementations, Apple failed hard by essentially pushing their Pentium III equivalent way past its sell by date and by getting a Pentium 4 equivalent hand delivered to them just as Dennard scaling slammed into a wall.
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:28:05 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     • Johnny Peligro
     @TURBORETARD9000 @Suiseiseki @mischievoustomato I'd like to see someone fully reverse engineer the Transmeta processors
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 00:28:06 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     • Johnny Peligro
     @allison @Suiseiseki @mischievoustomato Hey, I never said I knew how to fix the issues :akko_giggle:
     I just wanna play with it
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Wednesday, 07-Feb-2024 00:28:10 JST Sexy Moon

     in reply to

     • 翠星石
     • feld
     • vic
     @feld @Suiseiseki @vic I haven't kept up with the state of the art with games. It used to not really be a problem that VRAM wasn't shared because you just pushed never pulled. I assume shared is better because you don't even have to copy. But for AI stuff you're pushing AND pulling.
     feld and :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: like this.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:29:03 JST 翠星石

     in reply to

     • Iska
     @iska I cloned it, but I can't compile it without docker, which is kind of proprietary.
     
     At least it looks like the source-code of Ollama is there (but there's no real indication as to the license of each file, as MIT expat dumped into the root with no other details as to what it applies to doesn't mean much legally), but I'm not sure about the dependencies.
   • Embed this notice
     Iska (iska@catposter.club)'s status on Wednesday, 07-Feb-2024 00:29:04 JST Iska

     in reply to

     • 翠星石

     @Suiseiseki@freesoftwareextremist.com
     Just git clone the actual repo lmao and tell it to not install novidya drivers if it insists

   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Wednesday, 07-Feb-2024 00:29:19 JST Sexy Moon

     in reply to

     • 翠星石
     • :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:
     • Johnny Peligro
     @allison @Suiseiseki @mischievoustomato I was more just annoyed that Apple seemed to be incredibly dishonest about it (until they completely flipped when they went to intel) as far as the architecture was concerned yes power is sane
   • Embed this notice
     :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans: (allison@hidamari.apartments)'s status on Wednesday, 07-Feb-2024 00:30:53 JST :blobancap: :blobcattrans: :blobancap: :blobcattrans: :blobancap: :blobcattrans:

     in reply to

     • 翠星石
     • Johnny Peligro
     @Moon @Suiseiseki @mischievoustomato the funny thing is that the "megahertz myth" marketing stuff actually brought up a bunch of completely valid points.... which they subsequently completely ignored when they tried pushing the g5 everywhere, lmao
     Sexy Moon likes this.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:31:53 JST 翠星石

     in reply to

     • "; DROP TABLE Users;
     @TURBORETARD9000 AMD is only the slightest bit less bad than nvidia now - the VBIOS and peripheral software for their GPUs is proprietary and is cryptographically restricted to stop its replacement.
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 00:31:54 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     • Johnny Peligro
     @mischievoustomato @Suiseiseki I'm heading for AMD when I can, they've been showing better performance and thermals in their recent generations I've heard
   • Embed this notice
     Johnny Peligro (mischievoustomato@rebased.taihou.website)'s status on Wednesday, 07-Feb-2024 00:31:55 JST Johnny Peligro

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     i care about a small number of things, i choose intel for those things, plus there's no other x86 vendor I'd get
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:35:41 JST 翠星石

     in reply to

     • "; DROP TABLE Users;
     @TURBORETARD9000 x86 is a family of 16 bit to 32 bit microprocessors.
     
     AMD designed a new 64 bit architecture, but implemented x86 as a comparability mode.
     
     If you doubt my word, go and have a look at the non-marketing terminology intel uses to refer to their 64bit CPUs (it's not x86 btw).
     
     
     I'm pretty sure RISC-V implements many more instructions than architectures previously known to be "CISC ones".
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 00:35:42 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     @Suiseiseki In that vein so is x86, it's all RISC in the microcode (ish)
     It's all still x86_64
     Risc-V is RISC in the macrocode
   • Embed this notice
     ロミンちゃん (romin@shitposter.club)'s status on Wednesday, 07-Feb-2024 00:35:48 JST ロミンちゃん

     in reply to

     • 翠星石
     • Iska
     @iska @Suiseiseki llama.cpp werks good on cpu but it's slow
   • Embed this notice
     Iska (iska@catposter.club)'s status on Wednesday, 07-Feb-2024 00:35:50 JST Iska

     in reply to

     • 翠星石

     @Suiseiseki@freesoftwareextremist.com it's all a mess, I know :02_shrug:
     
     I also know llama.cpp exists https://github.com/ggerganov/llama.cpp and it doesn't seem to force Docker or cuda
     I did just remember wanting to run an LLM, I might use this

   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:39:25 JST 翠星石

     in reply to

     • Iska
     @iska >Compile with -D _GNU_SOURCE
     
     Maybe there's hope for this one.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 07-Feb-2024 00:39:36 JST 翠星石

     in reply to

     • Iska
     @iska >Compiles with -D _GNU_SOURCE
     
     Maybe there's hope for this one.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Wednesday, 07-Feb-2024 00:56:45 JST Sexy Moon

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     @TURBORETARD9000 @Suiseiseki riscv is apparently not well designed but it is open and easy to implement? that's how I understand it.
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 00:56:46 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     @Suiseiseki bleghh it's been forever since I read up on AMD/Intel history, yeah you got that much right
     
     RISC-V implements 40 instructions in its base ISA, 163 for RV64GCS (+40 if you count the compressed instructions separately), the minimum needed to run a general purpose OS
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 00:59:37 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     @Moon @Suiseiseki It has some quirks yeah, but I'm still in the implement stage atm with my visual core so I've got a bit of work to do before I'm ready to do a deep dive beyond reading the docs and figuring out how the hell to read verilog
     Sexy Moon likes this.
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Wednesday, 07-Feb-2024 00:59:50 JST Sexy Moon

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     @TURBORETARD9000 @Suiseiseki I remember chjara shit-talking it repeatedly
   • Embed this notice
     Sexy Moon (moon@shitposter.club)'s status on Wednesday, 07-Feb-2024 01:01:58 JST Sexy Moon

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     @TURBORETARD9000 @Suiseiseki they actually said why, which seemed to be based on actually looking at it. I just don't have knowledge in that area so I don't know if what they said was accurate. They're a pretty smart person.
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 01:01:59 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     @Moon @Suiseiseki I don't take much stock in shittalk without data to support it
     Especially considering how new Risc-V is compared to... everyone else
   • Embed this notice
     ロミンちゃん (romin@shitposter.club)'s status on Wednesday, 07-Feb-2024 01:26:48 JST ロミンちゃん

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     @Moon @TURBORETARD9000 @Suiseiseki
     >RISC-V having no provision for a condition code register (status register), or a carry bit
     which is... unfortunate for some specific applications
     Sexy Moon likes this.
   • Embed this notice
     ロミンちゃん (romin@shitposter.club)'s status on Wednesday, 07-Feb-2024 01:34:25 JST ロミンちゃん

     in reply to

     • 翠星石
     • "; DROP TABLE Users;
     @TURBORETARD9000 @Suiseiseki @Moon compare + branch
   • Embed this notice
     "; DROP TABLE Users; (turboretard9000@bae.st)'s status on Wednesday, 07-Feb-2024 01:34:26 JST "; DROP TABLE Users;

     in reply to

     • 翠星石
     • ロミンちゃん
     @romin @Suiseiseki @Moon Such checks are typically implemented with a compare, it adds an instruction or so but the instructions are cheap enough that it shouldn't be an issue (?)