It actually does affect people when you can't get medications and when you are banned from school sports. Suicides skyrocket. People get harassed and stabbed because it emboldens people who would otherwise suppress their urges.

It would be in the same vein as if black people were refused drugs for sickle cell anemia. It puts people in a lower class when they are treated differently than the rest of the population. I know the same fear and hate the people in my state felt towards queer people, because I felt it myself decades before coming out.