I don't think it has to do with 'sweat' per se.
I have to be attracted to his "scent". Bottom line.
Even if he is a handsome model and he doesn't have that certain right "je ne sais quois" scent for me, I won't bother. Likewise, if he is an average looking guy and has that scent, I'd be all over that, lol!
I can't describe what it is, but I know if I smell it.
Actually, this is funny you've brought this up. A while back, there was a T-shirt study in which they had about 10 male participants wear a T-shirt all day. At the end of the day, the people performing the study collected all the T-shirts and laid them out along a table, numbered 1-10. The 10 women participants smelled each T-shirt and picked the ones which smelled most appealing.
The interesting part is, after the woman picked a t-shirt, they were shown a photo of a guy who wore it and they were all 100% attracted to the photos of the men they'd chosen, BASED ON THEIR SCENT!
It can't be just anyone.