Even if you're in an area they currently cannot reach, there's no saying it will stay that way forever. One should be concerned, at least, if only for their own well being.

But, I can understand your rationale to an extent, I guess... There already seem to be plenty more willing to fight.