I have this same problem - my face breaks out if I use ANYTHING to wash it besides water and the hypo-allergenic Oil of Olay soap for sensitive skin (used to use the Dove, but they started putting sweet almond oil in it, and I'm allergic). I feel really badly, like I'm wrecking my skin completely, but I have tried so many brands across such a range of prices, all promising to be completely fine and safe and good, and none of them work nearly as well as my soap and water. So I'm sticking to it.
I would keep experimenting with moisterizers though - I notice if I forget mine, if I'm drinking a lot of water, in the summer my skin will be ok. In the winter though...forget it. My face gets extremely dry and tight, and I can almost feel the wrinkles forming. I've gone through the same very expensive trials with face lotions as I did with cleansers and only one works, without breaking me out/drying me out/clogging pores/giving me hives - Rachel Perry's Cucumber and Calendula face lotion. It's lovely and light, barely scented, a little expensive for me ($18 for 4oz) but it's something I can't do without. I would see if you can find some and give it a try, but the soap should be fine if you follow it up with a good moisterizer. I always read in magazines that you should never, never use bar soap on your face, but I think the people writing that advice do not have skin as sensitive and finicky as mine. I'd rather risk the drying effect of the soap than the acne, rashes, and clogged pores of even the gentlest, most touted cleaners (not even the Rachel Perry cleaners will work on me while the face lotion does, very disappointing).
Don't know if you have the same problem with the skin on your body, but I've found the Jason Cocoa Butter lotion (in the tube, not the jar - they changed the formulation in the jar and it's horrid) works perfectly.