When I listen to new music I select songs approximately based on:
* does it touch me emotionally, intellectually, is it really innovative?
* is well done, is arrangement, sound quality good, is it dancable enough?
* do I feel, think other people will really like it?
* is this song genious?
Then, I move it to contender list, which I listen from time to time. Now I'm more strict, and I'm listening to it as a dancer. If the song is genious, but not dancable, it will go to my personal collection or I might play it at 2am at house salsa party. The rest of the songs go to the dance collection.
Before I add a song to the set for particular night, I listen to it again, if I don't know it well. I might do some cuts, i.e. remove silence, chatter from begining and end, normalize volume, make notes.
Then I take prepared sets to the party, but I might or not play it. If it gets played I listen to it again and watch reaction of dancers.
From here the song will stay unless I feel I was mistaken or others don't like it. Or much better version of same song appears. If in doubt, I move it to purgatory for another listen, if I don't like it, but dancers do very much, well.. it hurts, but song stays.
Overall it's about 2-4 listens, at first I'm looking if it's good, then I'm trying to understand it, and then I listen if there is anything bad.