I think it's more an impulse driven by (naïve) conflict avoidant tendencies.

The real kicker, in my opinion, is that he thinks it isn't driven by utility, which most political decisions are. Obviously in this case, he thinks there's some higher utility in some abstract concept of "free movement" of people.