...if you disagreed with the President or current administration?
Most recently a few of the players of the Dolphins' 1972 undefeated season declined an invite because they disagreed wroth Pres. Obama's politics. Declining invites is not limited to this current president.
I wouldn't pass up a chance to meet a sitting President. It's not like we would talk politics...maybe the weather?