Remember everyone -
Although you may have had larger "follower" counts on the birdsite, not all of those followers will be active. In fact, a huge number won't be. They'll have just collected like dead wood over the years.
That birdsite follower count has a vanity factor to it. Ignore that - and look at your level of interactions here. That's the test of active followers and a vibrant community.
And then make sure you're chatting away with others too. It makes all the difference.