@ryanc Pretty sure they're still a thing. Or stopped being a thing pretty recently.
The IBM Blue Gene family for example was still being actively developed when I was an undergrad although I think it got axed when I was in grad school?
It was a distributed memory system like most modern supercomputers but had full custom interconnects for efficient MPI operations etc. And some pretty cool compartmentalizion and segmentation capabilities for running many small problems at once if you didn't need all... 32K cores for the BG/L we had at the time iirc?