It's just polite

Part of my job is to hold open doors. People come in, I open the door. People leave, I open the door. It's what I do. It's also polite for people to hold open doors for each other.
Now, as I hold open doors there are always two different reactions. The first reaction is of surprise and gratitude. Apparently, people don't hold open doors for anyone anymore, so this gesture (that I'm paid to do) surprises people.
The second reaction always, without fail pisses me off. It's when men, usually older, white men, stop and try to hold the door for me (um, dude, I'm not going anywhere, stop) or say something along the lines of "pretty girls like you shouldn't be holding open doors." For example, last night as I was headed into work I stopped to open the door for two older (white) gentlemen. I wasn't doing this because I was getting paid (I wasn't on the clock yet), but because it was polite. Rather than just go through the door, they had to stop, attempt to grab the door from me, and tell me to go ahead. They said, "No, sweetie, you go on in, it's chilly out here" with slight chuckles.
"No, sir, you go ahead, I already have the door," I replied.
"Nope, you go in, honey."
"Sir, I work here, and I'm supposed to hold this door. Go."
"So, since you work here you can't go inside?"
"No what? Fine. Whatever." and then under my breath "Fucking annoying sexist asses" The under my breath part might have been a tad loud.
What does one's genitalia have to do with whether or not they hold open a fucking door? Are you holding open the door with your dick? If you are, then yes, your genitals do determine who opens the door. But since almost everyone opens doors with their arms, your sex/gender doesn't matter.
I am not taking a swipe at your masculinity by holding a door for you. I am being polite. I am being a decent human being. Anyone can hold open doors. And anyone can go through doors regardless of who is holding them.
And as a side note, don't call strangers pet names. It's obnoxious and you don't me well enough to call me "sweetie" or "honey." In fact, most people who know me don't even call me such condescending bullshit.