Like Scott said, your pouch is mostly in your pelvis, so I think you are relatively safe in that regard.
However, I had a two step process, and I have taken some accidental hits to the former stoma/takedown site. That spot is still sore two years later. My fear is that some scar tissue will break off or decide to relocate itself.
maybe some rib protection device that wraps around your abdomen should be considered if your paranoid like I am about something going amiss during this activity.