You're going to have to hunt them down for the rest of the grow. A few seeds won't affect a plant but if lots of flowers get pollinated the plants will switch their energies to maturing those seeds and both yield and potency will suffer.
Don't bother saving seeds from hermie plants as most will just grow more hermie plants tho careful selection can breed the hermie trait out. I did that years ago with some Afghani Kush seeds a local guy gifted me from his large grow. The seeds I made from the non-hermie plants all grew fine and I sent seeds to half a dozen people at the time to try out.