unless it's literally covered in bananas, i'd leave it be - it won't make a single bit of difference to the rest of the plant whether it's there or not, and that branch, from pollinated area upward, will have a more full cannabinoid profile, higher thc content, lower resin production, and sweeter, milder flavor
it'll be more sleepy/dreamy relaxing to smoke than the rest of the plant too
probably