Is that your only plant? If so sure pluck them off and keep going. If you have other plants in the area just get rid of it. You really don't want that seeding the rest of your crops if you have more.
Get rid of it. Bananas...I can deal with and they usually pop up late flower. Balls are another story as they seed your crop much more efficiently. Idk why but it’s always seed city with them. They hide in places your eye can never get to. I have a plant I’ve ran for over a year that will throw bananas with sterile pollen. Never even the beginning of a seed. I don’t even concern myself with it.