I grew up in the Midwest where we are taught from a real young age to respect our elders.  We say “Yes Ma’am” and “Yes Sir” and all of that.  I entered the military and we were trained that respectful address includes “Ma’am” and “Sir” and all of that.  To call one the other is a real insult.  Sometimes someone says she don’t want to be called Ma’am.  Well, if she don’t like being a lady, then I’ll say “Yes Sir” and go on about my business.  With someone else.