It doesn't sound to me like liability concern, nor does it sound like withholding knowledge. A referral was suggested because the finding is outside their areas of expertise, and the right folks are a (long) car ride away. It's reasonable to ask them if it's okay to put it off as long as your pouch is functioning well. Docs do vary regarding how much of their workday they are willing to spend on email, phone calls, or other electronic communications, and that is partly a financial decision. For example, Kaiser docs get paid for email time, but most others don't.
Good luck! I hope this fistula and abscess/collection never give you significant trouble. If it were my pouch, though, I'd make an appointment with Shen. Dr. Remzi is reportedly not making appointments at this time.