IPFS links should be included in the activity as an alternate source of the content. It's very easy to do and everyone can do it without even running IPFS -- they just need the function to generate the hash.

Then if any server on the Fediverse automatically adds the media to their IPFS node during post ingestion it would make the IPFS link work for everyone else, thereby reducing the overhead of hosting media significantly