Anarchy is not about not following any rules whatsoever, it is about who decides the rules and how/by whom they are imposed.
I can think of a number of rules and restrictions that will quite naturally be followed, even at an anarchist gathering, simply because there is consensus.
Would someone wearing fascist paraphernalia be tolerated, for instance? Or would people behaving in sexist, racist, queerphobic ways not be told they need to stop or leave? Would it be tolerated to endanger other people with weapons and explosions?
So why should ableism be acceptable? Why should reckless behavior that excludes a number of people be fine?