@scathach @coolbean @elilla GMDH came out of a Ukraine lab in the 90's. Its a somewhat lost technology that grows one layer at a time, fits the current layer, and then prunes all but the top however many winners. Then repeats until stopped. Usually get small networks out of it that don't tend to overfit.

KANs basically turn everything in to huge b-splines, but there's some newer work (FastKAN uses resting bitch faces which apparently were another 90's era thing as RBF networks) about it. One of the funnier things with them is after training them on some problem data, you can try to match the actual basis function parameters back to known math functions. so its been used as a cheeky form of "symbolic regression" where you extract the math formulas out of a problem after the fact.

the black box nature of shitstacking multilayer perceptrons is kind of an open issue from the R&D side, it's just that the VC side doesn't care and benchmark chasing is measuring for that holy accuracy percentage at the complete cost of everything else.

:comfyderp: on the other hand i'm a weirdo who cares about things like compute, so i spend most of my time these days in neuromorphics and weird niche shit