Maybe it's the short growing season...
Hi again, I want to thank everyone for their suggestions and pictures :)
After researching this in the library, on the Internet, and here at the GW, I've decided a potager is not the best use of my space. I think it's my short growing season. All the best potagers have successive planting, so the gardens can be decorative and functional. With my three to three and a half months of frost free weather, my garden has to have everything planted at almost the same time. There are a few cool weather crops I can put in early, but almost everything else goes in the first of June. Since frost is the first to middle of September, this makes for a very short growing season. This doesn't mean I can't have a great vegetable garden, but I need a lot more space and bigger beds.
However, I love the style of the potager and I've decided it would be a lot more attractive if it was filled with perennials, for the most part. This way it would look like something besides dirt for eight months of the year (LOL) I've decided to put roses and herbs on the perimeter beds and some annual herbs and flowers in the four middle beds. Still planning on the "petunia fountain" in the middle and the purple raspberries on the outside of the fence closest to the pasture. I love the idea of the structure of the potager and I think the fence will (hopefully) help keep the deer out. They also don't like most herbs, so we'll see how successful I am. (The deer aren't hungry and ignored my vegetables this year completely, they just like snacking on the roses.)
The vegetables will go into some beds in the back yard (I have a small back yard with most of it in pasture) and a bigger garden beyond the potager for freezing, canning and drying vegetables. You have to grow a lot in a short amount of time to get through our long winters :) Thanks for the advice and help and feel free to contribute any other ideas! Thanks again :)