as long as they are gathering light (feeding the rest of the plant), i'd leave them on. two exceptions:
1) if they're diseased, moldy, thrips, whatever - get them off (and check the rest of your plant for similar issues)
2) towards the end of veg, i have found that stripping the bottom third...