The idea of the Fediverse is to federate with other servers, to allow people from many different servers to communicate.
If you want a closed community, it might be better to use a non-federating platform? For example a @nextcloud server lets you have closed discussions restricted to members of that server, including chat and audio/video conferences using Nextcloud Talk.
Having said that, you can run a Mastodon server in isolated mode too: https://fedi.tips/creating-an-isolated-server/