I think that if you do pickup a little noise during a take, you should consider the take as if the noise isn't there. Is it a great take? just ok? could you do it the same over again? better?
If you can do it better, or the same. Throw it out, eliminate the source of noise and just do it again.
But what if you can't do it again? You tried for half an hour and never nailed it just so again? I say keep the take with the noise. Of course there are degrees to how abrasive the noise is, so use your best judgment.
In the end, the recording I was working on just wasn't that important and I was pressed for time, so I kept the noisy takes. I was also going for an indie low-fi sound, and I considered adding some extra noises to the song as well to further the mood. If you're interested, you can check out the song.