Fellow introverts, do you find it hard if you have to be around family a lot? Any tips for keeping my mental health in good shape?

Ever since B was born, we've spent every weekend in long family visits. Before, we'd just visit my parents or MIL for a couple of hours on our way from work, but with a kid it doesn't work out that way so we have to go on weekends and it turns into these day long visits. Plus our extended family wants to see him, so we go and visit them.

I'm a constant ball of grumpiness. DH and DS are very outgoing, so DH really doesn't understand why I blow up every few weeks and tell him that I'm sick and tired of seeing family and will be happy to see them once a year. Plus I'd really like the chance to take B to a museum or sign up for pool time at the local Y instead of spending my days yakking. I just can't find the right words to make him understand that even if I love these people, it takes a lot out of me to see them so much. And that I'd love some of them more if I was around them less