Yes, it will affect your other plants. If the pollen sacs burst, you'll end up with seeds in the sours. Carefully remove any balls, and stay on top of it. If you can move the hermed plant to a different area. The bud will still be ok, just a lot of extra work staying on top of things.