If it went viral it might have an effect, but so far it doesn't seem to be a problem?
(An alternative approach is to upload the audio onto a PeerTube server, as the files there use peer-to-peer technology to spread the load across everyone who is listening/watching audio or video. Plus people can follow PeerTube accounts from Mastodon etc as they're part of the Fediverse.)