Why is the penis/vagina thing the be all and end all for men? It’s just so shallow and really sort of yucky.
Every woman can tell you the story of the moment they realized that they crossed that point from being a child to becoming a sex object for men. When the men around them regardless age looked at them and treated them differently. I'm sure it was creepy and weird, but women also realized they had power and that their sexuality had enormous value.
If you really believed PIV sex is no big deal, you would have been handing it out like candy at Halloween, but you didn't. You protected it, like most women. Why give away something that has such huge value for free, right?
Men have something of value too and that is their commitment. Ten cc's of semen cost a man 25 calories and is nothing, but choosing the wrong woman to marry costs big. That's why this conversation is pointless because I don't expect you to wrap your head around the concept monogamist commitment means more to a man than a woman.
Even while I was married to the laziest woman in the world, where she put forth zero effort in to our relationship, I could always hang my hat on, "Well... at least she is faithful." Your husband thought the same thing too and regardless of whatever issues you might have had, he thought you were special.
That special-ness ended when you rejected his commitment in the very concrete way of having an affair. You will never get it back. I know you want it back. Your husband desperately wants it back. But it is gone. Poof.
So the short answer to your question is: It isn't the PIV thing that men are hung up on, it is the complete rejection of the commitment your husband gave you that is the issue.