Yeah I have half of my ductwork insulated, but I need to finish off the other half because it really does cut down on the noise a lot.
What you want to do to quiet the fan down pftek is hang it from the ceiling or wherever on bungee cords so that the vibration is not transferred over to the wall it is hanging from. Let me see if I have a pic some where as lights are off now and I can't actually take one right now....oh here it is!
Note how the fan is basically 'hung' on bungee cords. This eliminates the vibration transfer. You can also see my $20.00 Harbor Freight Tools 'router speed control' AKA 'fan speed control' hanging in the pic too.