@IAMAL_PHARIUS @Codeki @PopulistRight @snappler @RustyCrab @Inginsub I think what's especially pernicious about the scheme is the complete lack of apparent elegance in the hardware solutions on offer. Nvidia's original tensor cores *happened* to be good for AI, but their actual capabilities were clearly targeting scientific/HPC. There's been scarce improvement in any area except deepening the block.
There are plenty of ASICs which outperform nvidias offerings on a per watt basis, but they cannot compete with Nvidias wafer purchasing power, or the very wide industry adoption of pytorch and cuda. Nvidias shipped libraries have been proven to be inefficient and haphazard and doing things the nvidia way, while typically good enough, is rarely the right way to extract maximum performance. But you can't really tap into the available resources without blindly relying on Nvidia's "trust me bro" middleware.
I call it pernicious because the entire industry buying into this one vendor is pigeonholing millions of man hours of engineering into nvidias black-box toolchain.
Nvidia knows that this is a gravy train. They have no intention of meaningfully innovating fixing their flawed foundation. Maybe its partly because doing things right could break over a decade of CUDA and now tensorflow work, but I think it is more likely that Nvidia is in a downwards talent spiral. Nobody there now worked on laying the ground work or architecting the original IP blocks. Nvidia's total reliance of proprietary code for their value add features and workflows means any deviation from their current path is a massive Pandora's Box.
I know this thread is more about the "AI" side of things, and how the infinite money printer depended on infinite growth in specifically needing Nvidias miraculous one true computational solution. And how it turns out nvidia is as much in the way as it is helping. However, I thought it was a good idea to dredge into the past and point out that the entire valuation of Nvidia today is built on an IP block that *happens* to be *competent* at AI, and at this point NVIDIA can't risk a pivot. I know that the models coming from China still used Nvidia, but the first chink in the armor was the fact you don't need a gigawatt of GPU power to compete with o1. I think the next one will be an IP block specifically built for this kind of work - likely a hybrid FPGA-in-memory+matrix ASIC-in-memory, basically be able to constantly propagate forward in memory by staging FPGAs rather than go back and forth between memory levels and program levels. Nvidia can't up and invent any of this, they're already behind and even lost their ARM bid.
You'd think that companies spending hundreds of billions on leadership in a field would not want their entire business model to be hinged on a single vendor lmao