@tero it might just be working because most networks are too dense anyway. stanford's butterfly matrices capture more or less every standard transformation matrix. then there's the work in sparsity numenta's complimentary sparsity, or topkast. GMDH grew smaller networks incrementally. numenta's cortical models used to nod about how they could subset engrams and they still mostly worked.