@jenniferplusplus @rimu Thanks for your answers but I think we mean something different. My concern was about undoing the "Accept of the Follow" not the Follow. The point of view is the actor receiving the follow, not the actor following another. I think one should not send an Undo of an activity one hasn't issued oneself.
FYI: I found this answer interesting: https://fediversity.site/item/900328af-5252-4198-aa8b-b97bd541bbb8