Yeah, there are a whole lot of alternatives to using soy, although it hardly sounds like you're eating a huge amount. There's rice milk and oat milk and a whole lot of other things that can be ground up and turned into a "milk" (almond milk, anyone?).
From what I can recall, not all breast cancers are estrogen sensitive, so avoiding soy might not even be an issue for you. I'm sure your oncologist can help you figure this out. Heck, I just read the other day that soy products aren't estrogenic enough to prevent menopausal symptoms!
Take care of yourself, and don't let your mind make your body feel bad!