i prune for shape after it blooms. have done some heavy pruning on my plant, taking off branches up to 1" dia. from the base on up to 3'-4' from the ground. the mid-level branches tend to weep slightly downward anyway and cover any exposed cuts so i am left with an arborescent shrub. they seem pretty tolerant of pruning, in my experience. could be just a slight difference in our zones but i have not had any winter dieback on a 20-yr.old plant.
two years ago, one of the dominant branches in the top of the shrub was broken during an ice storm. i cut it below the break and it started filling in nicely the next spring.

Carrie,

I would be more cautious than Jeff, probably overly cautious. I carefully decide what branches to shorten. I prepare some melted organic wax, such as those from a bee hive. After the pruning, I carefully seal all the cutting surfaces with a thick layer of wax. This is safer if the tree is weak and small. My plants like this treatment :-)

Have you tried the beautiful red-flowered manglietia insignis? Those can grow to 100 feet tall.