Sounds to me like you're being a little oversensitive if minor, probably unintentional, only arguably low-key racist comments come off as "really terribly".