Yeah, people will think you're an asshole for not going. And that's too bad. It's a tradition and basically even if you don't like going, even if you're mother understood this, you're going to be seen as a spoiled brat for not sucking it up and showing your 'respects'.