How about rosemary. It is evergreen, does well in a pot and can be pruned to whatever shape you like. Then each spring you could plant red or purple verbena or red or purple petunias to trail down the side.

You can borrow a book on container gardening at your local library. Or visit the container gardening web site here at GW.
I like the look of various plants in a container. Something tall & slim in the back, something full with interesting foliage in the center, something cascading in the front. I found this today, it is a picture of a grouping of containers, along the lines of what I mean.

Here is a link that might be useful: 2006 Container Pics

How much sun will your pots receive? Will they be in the sun all day long or do you have a lot of trees around, so they will get a good bit of shade?

I have two pots on each side of my walkway. They will get sun most of the day because they are facing the east and there are no trees to shade them. I kind of wanted something that would come back year after year. I like the purple or lavender colors.

I agree with the suggestion about Rosemary, also maybe Purple Heart Vine, aka Wandering Jew, which does wonderful in pots in the hot sun. I don't know how good it would look in pots, but 'Butterfly Blue' Scabiosa has flowers that are a really pretty lavender color and it blooms almost all year long (mine only stopped blooming for about 4-6 wks in Jan.-Feb.) and the foliage is evergreen.

The difficulty with perennials (that come back every year) is that they tend to have short blooming periods vs. annuals (that are only used for a season). So where most perennials will only bloom for a few weeks (some longer, some shorter), annuals give that pop of color for a few months straight...which is why they are often used in pots and planters and changed in spring/fall.

I'd suggest that when you go shopping for the plants, I would grab a pot that kinda looks like yours and then place the little pots of the plants that you're looking at inside to kinda get an idea of what it would look like.

Here is a link that might be useful: Purple Heart Vine