The little white speckles kind of resemble the first signs of spider mite damage. Inspect closely under the leaves with speckles to check for mites or other bugs.
What nutrients are you feeding, feed/water sched, pH, ppm/ec? What kind of temps, humidity are you getting where you're at?
By this time in the season on my outdoor, quite a few of the lower, old fan leaves turn yellow, and my plants are well fed for sure. I think it is the plant reusing nutrients from the older leaves in new growth/bud development. Every day, I give a very light pull on the yellowing leaves. If they don't let go, I check them again the next day. If they do let go, then I toss them in the compost.
Good luck! Peace