Why don't tax evading Corps just leave, they can be replaced + we can put heavy tariffs on their exports to US?

Good idea. Some people think corps make money out of thin air, they use the resources of the US, and our technology to get rich.