Janne Moren
@dymaxion @suzannealdrich @cstross
I was going to say the same thing: none of this hardware is usable (never mind useful) in a homelab.
With that said, 3-5 years is, I think, not unrealistic. We have racks with 8-9 year old GPUs and they're still in constant use today.
While it might not be used for training, they could still be used for other tasks, or find second life in academic HPC centers.
With Moore's Law on its last legs, they will likely be useful for a long while.
mark
@jannem @dymaxion @suzannealdrich @cstross I am currently working on the possibility of removing a lot of A100s which are only 3 to 3.5 years old that were created just after the start of gpt/llm fever. The gb200 / 300 are just that much better, and there isn't enough floor space available quickly enough. Its a crazy race.
So ya, these aren't really home user level things, but they're pretty damn powerful and still air cooled.
Eleanor Saitta
@suzannealdrich
The infrastructure needed to keep these machines happy is nontrivial. Each single server hosting eight graphics cards is north of a half million new, so even at ten cents on the dollar they're the cost of a new car and they're mostly designed to be used in clusters of at least eight and generally more like 64. They only have boot drives in the machine, if that, so you need an external storage server. The network cables they use run $600 for 3m. The switches run into the six figures too. The new hardware is designed for rack scale water cooling, so you're going to need to build a car-sized heat exchanger unless you want to literally just run all the faucets in your house wide open all day. And a half rack is still going to draw at least ten times what the entire rest of your house does.
@cstross
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.