Harm is a pretty solid metric. Not some imaginary "think of the children" harm, but the "this disturbs/literally harms me" kind.
Yes, some people are precious little weirdos that won't want to see anything. The question then falls to society to determine if it was ultimately tolerable if they bring up grievance. Then the paradox comes in because the general vibes are always a moving target.
A gay person existing doesn't actually literally harm anyone though. A homphobe shouting slurs at a gay person, excluding them from vital social, economic or whatever activity or beating them up does very concretely harm someone. It's not that difficult.
The problem with this is that people disagree about what harms others. Right wing insane people are not living in the same reality that you and I are. They genuinely believe that even seeing a gay person is harmful. They genuinely believe that the existence of gay people is harmful to others.