Granted it may be inappropriate for those of a harm none disposition but generally I can't see the point in having power if you don't use it. Manipulation is a means to an end and providing you can justify that end (and when I say justify I mean align with your own moral spectrum not justify to another individual or belief system) then to me it is a natural function of performing magic.
I'm not sure if I'm honest how I'd function as a witch if I didn't manipulate people, it's pretty much the essence of my craft. And as I'm not great at manipulating people on a mundane level its natural I should turn to using the skills I am good at. Do I feel guilty? No not really, it's a little selfish of me perhaps but I work toward my own desires and I think you will find most witches do.
Incidentally some people are far easier to manipulate than others. Even unknowingly there are some individuals who have a real resistance to magic. Others - just a little nudge will send them scuttling to do your bidding... I adore the latter, they make my life so easy!