You can trim almost all your leaves off and the plant will survive. How much is too much? That's the argument. No plant is really "damaged" by removing a leaf or two, but you may consider alternatives like LST or just trimming the lower, tiny buds/branches to direct energy to the ones that are going to be your large buds. Take into account the size of what you're working with and go from there.