I don't necessarily agree that a slim silhouette is always better.
Done properly, short people can do baggy like nobody's business , while a very tall person in baggy clothes often just looks clumsy.
Today, I met the
coolest petite woman at the hospital

: a woman doctor in her 60's (very slim and tiny), she wore a ginormous white doctor's coat in what was obviously a large man's size, letting the shoulders droop to her elbows, sleeves rolled up generously and collar raised. The back billowed dramatically as she walked, giving a glimpse of black drainpipes and pointy ankle boots. While the others in the uniform just looked like frumpy doctors, she alone could have been in Yohji.

I don't think it would have worked on a very tall person.
I also have a guy friend (a successful designer), who is very short and muscular. He always wears his Armani suits and shirts way too big, but it's charming and stylish, in a Chaplinesque way. I think if he wore a well-fitting suit, he'd just look too pert and normal and even smaller.