In my experience, the world doesn't particularly care whether we exist one way or another - there's no preference, per se, only a reaction to the changes we bring about. Within that line of thought, it's not us that the world is inherently displeased with, but rather the fact that the natural order of things is being upset.

And really, I find that I really can't argue that time and space displacement isn't upsetting the natural order of things.