No, because it would cause complications if someone had replied to you who was no longer allowed to reply to you.
For example if you start out in Public mode, anyone can reply.
If you changed it to Followers-only, then this would cause problems for non-followers who had replied to you.