Actually, the masculine physic has more bulk to it than the female (depending upon the individual, of course). That's why Michelangelo prefered the male body so much more than the female. He liked the weight to the male form, and even enhanced it to fit his ideals. If you look at his sculptures of women, you'll notice that they basically resemble men with breasts and without the genitals.
Similarly, if you look at the differences between male and female bodybuilders you'll see why they don't compete at the same level. Just as women have had such a difficult time getting into firefighting, another male-dominated proffession. The reality is that women are naturally of a lighter build than men, and clothing over the ages generally fits with that idea.
Of course, fashion is often about the distortion of human form, which is why we get corsets, bustles, etc. However, your perception of female fashion totally ignores entire segments of fashion history. The Greeks and Roman, the entire Empire period, the '20s... All focused on a more authentic female form.
Just look here, please.

Joan of Arc, slightly muscular, but still a woman, wearing a man's armour. You can clearly pick out how the harder lines of the armour don't parallel her form.
http://maidjoan.tripod.com/swynnerton2.jpg
Anyway- I'm going majorly off topic!