It's not actually a real dilemma, whether to Not Judge and take the moral high ground, or to do the sane, safe, and smart thing and limit the amount of things you're uncomfortable with having around you. They're really very different operations. There's a distinct difference between saying, "You're a bad person and I hate you and you'll suffer for this!" and "From what has been happening, I don't think you're doing/involved with the sort of things I want to be around."
It's not for us to judge another's heart and motivations against good and evil (although we may be tempted in extreme cases, see "Godwin's Law"). But it is for us to judge a person's actions against "things I am willing to be around and put up with." The first is a divine privilege, or only done within one's own heart. The second is not just a human right, but a human responsibility.