@darius @charlesrandall I think I agree with @evan 's sibling comment the thread, it's not hard at all.
The server where the object that is being replied to is the place where this enforcement can be done. An object can claim that it is inReplyTo another object, but the server of the original object has authority to act on this fact. If the server doesn't add it to the replies collection of the original object, then it's other instances should not see it there. Am I missing something?