@silverpill in the context of Mastodon's FEP, perhaps cache poisoning could be a problem, but from the context of ActivityPub, there's no "cache poisoning" because the canonical version of an object is the one the server retrieves explicitly based on its ID, anything else is hearsay and should not be trusted by default.
Also I find very hard to reason about what you call "cache poisoning" when the entity that serves the object is the same with the one that sends the activity.