Now you are doing something on behalf of others.
If an individual used a tool like, say, blocktogether and they were already following 3 people on the list, blocktogether would recognize the _follow_ (not just an approval) and not block them _for you_ while leaving them on the list.
This sort of thing makes for a powerful mitigation tool. You can mitigate the harm to your social networks of false positives.
But when your server admin makes that call for you your only recourse is to move.