I've had these a lot. Tylenol never helped me either and I definitely can't take Ibuprofen or drink coffee because of IBS. Plus, I would be concerned that withdrawal from caffeine would just cause more headaches and using Ibuprofen too often may result in rebound headaches.
Really hot, steamy showers have provided the best relief for me, especially when the tension headaches come with a stiff neck and shoulders. I'll also have a cup of strong chamomile tea (two bags instead of one).