Why am I Embarrassed?!

Last Friday I was unfortunate enough to have experienced what is, quite possibly, the most upsetting, offensive situation of my life to date. Long story short, my boss offered to take me to lunch with his friend because, well, I'm pregnant and I like to eat. So I'm all, "Hell yeah!" So we go to lunch up the road at a bar/restaurant and come back to the office all stuffed and happy. My boss then takes the friend we went to lunch with to pick up his car from the shop where it was being repaired. While my boss is gone, his wife arrives at the office and starts flipping out on me for going to lunch with her husband. Long story short, apparently their marriage has faced some issues in the past and they had an understanding that he was not to take women to lunch anymore. (Apparently, I was somehow supposed to know this...) "Okay," I'm thinking to myself, "while she's totally out of line for yelling at me about this, she's clearly got some past reason to be suspicious of her husband's actions in this situation, etc., etc." So, I'm just letting her vent and kind of blowing it off. Then, suddenly, she's in my face yelling about how I'm a married woman going to lunch with a married man, and I should be ashamed of myself. "For going to lunch?" I asked. I'm still so dumbfounded at this point that I can't wrap my head around what she's insinuating. Finally, she crossed a line. "I want you to look me in the eye and tell me that's not his baby." ...I'm going to give that a minute to sink in, because I know it took me a second. Now this crazy woman has gone from making accusations about her husband's intentions to insulting not only my fidelity to my husband and marriage, but my unborn child?! Ugh! I can think of a dozen things that I wish I had said to that woman now, but at the time I was so blown away that I just looked at her and said, "Are you kidding me?!" (and, alright, I might've embellished that with a few curse words).
My boss returns from dropping his friend off, his wife turns and starts lashing out at him, and my boss turns to me and tells me that I can leave and he'll pay me for the rest of the day because I don't have to sit there and listen to their nonsense. Later that afternoon/evening I received several emails from my boss apologizing for his wife's behavior and stating that he could assure me that something like that will never happen again and his wife will more than likely never set foot in this office again. I replied to him letting him know that as far as I was concerned we were cool, but if his wife ever came at me like that again, it would not end well (again...that might've been said with more colorful language).
So, this past week at work everything has been surprisingly normal. I think we have both just decided that we'd rather not address the issue and let it consume any more of our normalcy than it already has. After all, neither of us have anything to feel weird about (especially me).
This whole situation has gotten me thinking, though...why in the world was I embarrassed about telling people what had happened? I haven't talked to my best friend about this, I haven't said anything to my mom. It got me thinking...why, when we are victimized, do we feel ashamed of telling people about it?
So, I've decided that if this escalates any further (and quite possibly even if it does not) I will be looking for a new job. I cannot come to work every day wondering if some woman whose marriage is apparently a complete sham is going to pick today to come in and take it out on me.
I'm not looking for sympathy or advice, just really venting.