A Michelia figo has been growing in my zone 7b garden for several years now.
Don't grow Daphne odora. According to Dirr, it's temperamental or as stated in the link, persnickety. Hardiness Zones: 7 to 9.

Here is a link that might be useful: More about Daphne

Thank you Terramadre. Any experience with sweet shrub? Mock orange? Any other fragrant shrubs I'm missing out on?

Yes, Michelia figo grows north of you and sweet shrub does well too. It's native to the area.

i have about a dozen or so different michelias in my garden. all are fragrant. you should try michelia platypetala, maudiae, and skinneriana. there are large (12-18 ft) figos in the anniston area.

i have a 4 x 6 ft variegated daphne odora...the largest ive seen. its growing on a steep slope in part sun. the soil is rich, organic and well-drained.

also try tea olives (osmanthus), some camellias like 'fragrant pink', texas mt laurel, and chinese abelia.

Tea olive definitely grows north of Montgomery. There are plenty in the Birmingham area. Most (butterfly) coronarium gingers grow well in Birmingham. I have 2 or 3 varieties. I received a curcuma ginger in the pot with a trade and it comes back and bloomed this year. Gardenias are not a problem. If we have an extended cold spell below 10 degrees you may have some leaf loss and even some die back but they will come back. Sweet shrubs are native and grow even farther north than Birmingham.

I've been growing both Michelia figo and Daphne odora for several years, and I'm 25 miles North of Birmingham. I even transplanted my Daphne successfully from my previous home. Like plantaholic's, mine is in rich, well-drained woodland soil. It grows very slowly, though!

My largest M. figo is about 11 feet tall and several feet wide. It's one of my favorite plants in the entire garden.

I don't know whether I grow them or not. Don't know the Latin names. When I was a kid we used to gather sweet shrub blossoms from the woods in DeKalb Co,which is about as far north as you can go in Al. I had mock orange in my yard in St. Charles Missouri many years ago.