Why is this such a problem?
Because like all tech billionaires he has an agenda which is incompatible with the ideals of open source. Giving him a platform where be can speak about his interest, despite not having contributed to the community is acting against the interest of the open source community as a whole.
You wouldn't invite a fossil fuel lobbyist to a renewable energy panel either, would you?