I only ever take it off to bake, like when I'm handling dough. Or I guess if I'm making something that is pretty hands-on.

Otherwise I never take it off, not to shower, gym, swim etc.

Oh, last summer I did take it off when I went tubing.