Multiple things going on: some immigrants (and especially the labor contractors hiring them) are being brought in to work at lower rates of pay and restrictions than native workers - i.e., exploitation. Then there's nativism, prejudice on skin color/religion and so on. All which can be turned into wedge issues to rally people.
Sadly, what with destabilization, climate issues and so on, this is going to get worse world wide. And it's far from a US only issue.
It sucks.