This is usually considered risky because of the chance of bringing pests inside. Outdoors, there's a whole ecosystem and weather and stuff to keep their numbers under control, but indoors there's not. A few mites or thrips on a plant can quickly turn into an infestation. Not saying it can't be done but you're taking a chance.
Starting indoors and putting outside to finish is a different story, and perfectly fine to do.