Morals aside, ultimately the less Muslims their are in the world, the less radicals (Muslims who follow the Koran literally, word for word) there will be ,therefore the world will be a safer place.
This doesn't mean all Muslims are violent. Also you don't have to fear something to dislike it or...