If the leaves are just flicking away, its fine, if the breeze is too strong, the plant will tell you, you'll see that it's struggling to hold its 'head' up, if its not struggling, its fine.
Put something infront of your fan to direct the breeze away from your plants, but in such a way that its...