Only one of those operates by force and will lock you in a cage if you disobey.
I agree, people coming in is not the problem. But who decides when, who, where and why people come in is important. Right now, that's the state.
It brings people into its national boundaries and than gives out goods and services paid for by others, creating resentment.