I think most drugs if not all will affect immune system because Crohns and ulcerative colitis is a auto-immune disease and medicines work to affect the immune system in order to calm down the immune system and keep it from attacking healthy cells/tissues, I do not believe there is a drug that treats IBD that will not affect your immune system sadly.
I would suggest wearing a mask more often, I see a lot of people doing that to avoid getting sick.