I only feel guilty in a few circumstances
First is when I'm feeling stressed and I buy something just to comfort myself. I don't buy random things, but I can forgo the limits I've placed on buying an item. I'll promise myself that I won't even think about getting X until it is marked down again, but if feeling down, I'll buy it without regard to the cost. I always end up with items that I love, but I know their purchase is only a temporary fix. I could buy 8 suits this week. By next week I'd have my eye on something else. That's why I try to space out purchases. Rather than getting three things at once I'll get one or two at a time when that's an option.
The second time is when I've just purchased something that I like, but don't necessarily love, and then, within a few days, I happen to find a must have. I usually end up getting it too. If it's something ordered online, I'll spend a day or two thrilled to have discovered it. Then, while waiting for the package I'll start to have second thoughts. Then, when it arrives I see how much nicer it is than it looked online and let go of all of the guilt.
As I mentioned, I avoid a lot of these issues by getting one or two things at a time rather than splurging. A good example was about 6 months ago when I was coveting two grey Margiela blazers. I knew I should only buy one, but was having such a time deciding. I ended up basically flipping a coin and telling myself I'd buy the other later if I was still into it. A few days later I visited the Saks outlet and found some deeply-discounted prada loafers. If I'd bought both blazers I would have felt incredibly guilty about the shoes. But, problem avoided.