No. Letting it sit overnight does not distill the water only evaporates the chlorine, ph should remain whatever came out the tap I bet at least 7.5.
Distilling water is just like distilling spirits.....only the spirits use a mash instead of just water....outlined here:
Honestly the chlorine in your tap is fine, you might let it evaporate but the true test as to weather or not to use the tap is going to be ppm as in parts per million and of what?.....and PH....which can be adjusted with ph down, lemon juice or whatever.
Then it might be chloramine as well.....which is no biggie either if you know what to do about that, I don't but recall a post from member hyroot yesterday that outlines what to do there....I will try and find it.
Honestly I would just recommend the machine water....the ppm should be zero and ph should be 7.0......if this becomes a big hobby for you then you might consider the RO system in your home.