It doesn't matter if we weren't raised as girls. We've always experienced gender based violence.
But I always say, "they bullied me for being feminine, and called me a girl (as if it that alone was an insult), but once I transition they call me a man?"
Make it make sense.
More than that, when you live in a female role long enough beginning at any age, you get a crash course on the differences in the way women are bullied.
I transitioned 18 years ago. I've lived far more of my adult life as myself than that other person. So why do I need to hear criticisms about how I didn't have a 'female childhood'? It's not like I internalized patriarchy and didn't speak up and even volunteer to protect women's rights.
Yes, my role models were both male and female for different reasons, but when your surrogate father is Jean-Luc Picard, you probably identify with the virtues necessary to take patriarchy to task... and I did. Regularly.
The problem comes in that there *are* people out there who fetishize womanhood and their usually closeted, shameful expression betrays any attempt to hide the fact that they only see woman as sex objects.
So your wife is right, too. There are people out there who literally see only the clothes. Both things can be true.
Many say it's "transphobic" to make this distinction, but I'd argue that transvestitic fetishists (AKA AGPs) are not trans. Maybe some are, but as a group...
Unless they learn a lesson about the experience of womanhood beyond the sexual, this is a paraphilia, not an identity. It deserves First Amendment protection from other men, sure, but it doesn't deserve respect from women, cis or trans.
The biggest issue with the extreme polarization around trans issues is that it's one of the most nuanced topics in all of human existence, and we really need to learn to communicate that better.