Some of us do have food sensitivities, but I think the variability you describe is more often the randomness of small intestinal output without a colon to stabilize things. When you say "in the bathroom all day" it's hard to tell whether you're able to empty your pouch properly, whether you're having urgency or not, and whether your stool is liquid or not. Gas is also sometimes the issue. In any case, the most general solutions that many J-pouchers employ are fiber, bowel slowers, and "stretching the pouch." I don't know if you're using/doing any of these. Most commonly for fiber people use psyllium (Metamucil or Konsyl) a couple of times daily, to bulk up the stool and minimize the watery output that can make life harder. For bowel slowers people use either Imodium (non-prescription) or Lomotil (prescription). I prefer these mainly at bedtime, but some folks use them more frequently during the day. For "stretching the pouch," most of us are advised to gently and gradually delay bathroom trips to increase our pouch capacity, which may have more to do with learning to put off using the toilet than actual pouch volume.
Do you have a pouch-knowledgeable gastroenterologist, or continued care by your surgeon? What does your doctor think is going on?