You are right: We don't look at infertile people that way, because, as you also mentioned, reproduction is not the only reason for sex. An infertile person can still be a wholly satisfactory sex partner, whether long-term or short-term.
Again: People are upset by an individual who presents as being of one sex when they actually are not. It is deceptive, whether one thinks it should be or not - but how could it not be? That's kind of the whole idea!
This is why "disclose before dating" is vitally important, not only to protect the trans person but out of basic human respect for the one they're dating.
Anyone see the movie To Wong Foo, Thanks for Everything, Love, Julie Newmar? This very subject was brought up and I'm always reminded of Vida Boheme's (Patrick Sawyze) speech about "human rules."
I'm afraid that human sexuality is not going to go away, no matter how much people would like to pretend it doesn't exist so they don't have to deal with it. Part of life as an adult IS learning how to deal with it.
ETA: I don't want to appear entirely unsympathetic, though. Over the last decade or so - maybe longer - the normal boundaries that have been in place for generations are being torn down and that has caused tremendous social problems IMHO.
Hookup culture is seen as the normal and healthy way to get sex. Very often it's with strangers (boundaries shot) or if not, it's with friends (normal boundaries torn down.)
Men and women live on the same dormitory floors at college. Anyone who objects to this is a prude, or trying to oppress others, or somesuch.
These are only a couple of examples. There are many more that make it quite understandable as to why someone would want it all to go away, as the poster above is essentially saying.
Allowing individuals who are physically of the opposite sex into the privacy of restrooms is, again, a loss of normal boundaries for many, many people. But they'll be subjected to a barrage of ridicule and name-calling if they dare object.
I submit that this is not about "hate." This is about boundaries - boundaries that are being torn down from every side to the detriment of all and the understanding of none.