I have a 4x4x8h tent in a room with 4 plants. I'm running a 400 watt light so I'm venting and at the end of week 4 veg the place was already getting that "pot" smell. When I closed the door to the room it masked it but the inside room stunk.
You might be able to get away with it, but paying the 160 for a fan and 80 for the carbon filter that killed ALL smell was worth every penny. Peace of mind is well worth what you might spend on it but with 4 plants and growing with CFL you might be ok.
GL with your grow.