Yoga is simply a form of meditation that includes low impact exercise. Anyone can meditate and exercise, so doing the two together is no different.
I cannot stress enough that in Paganism, there is no “can’t” or “shouldn’t”, “must” or “have to”. You do what feels right for you, the opinions of others don’t matter. If Yoga helps you, physically, mentally and spiritually, then you do it! To heck with everything else!
Also worth noting that yoga can be an excellent mental training for focusing the mind. This kind of mental strength and direction is a very useful skill to have when practising any kind of craft path.