Peak potency is hit once all trichomes have turned milky white. It retains that level of potency until all trichomes have turned fully amber and after that potency decreases.
 
So if you want peak potency you can harvest once all trichomes have turned milky white or you can wait until part are milky white and part are amber or wait until they are all amber. What you need to do is to decide if you want slightly more of a head high or slightly more of a couch-lock stone or a combination of both and then harvest when the trichome color will give you what you most prefer.