They fled to a different province because the person persecuting Him didn't have power there. If they were or not inside the Roman empire during that time is not relevant for the story, only that they were away from the reach of Herod, but it becomes relevant when people who hate both Whites and Christians want to use it as a tool against Whites.