I'm pretty new to vege gardening, and am just starting to put together a potager.
You can do ALOT in a 20x20 area.
The brick wall has me worried that you may, indeed, get too much shade in your small space. You need to study your yard and see where the sun hits at what time of day, and decide if you want a winter garden also.
Mine is on the South side of the house, so it gets pretty much full sun til late afternoon when some trees shade it.
I don't have deer, but I try to keep my eyes open for ideas for my sister, who has tons of them!
I seem to remember that you angle your fence out from the garden so the deer can't jump in. They don't like the angle or something _/
I wouldn't plant fruit trees outside of the garden area if you actually want fruit and/or trees that are alive due to the above mentioned deer!
I'm sure you'll get much more help from people who live in deer country, but Hope this helped a little.
Nancy

It's possible to figure where the wall's shadow will fall at different times of the year, and plan your beds accordingly. I did this with my potager, and found that the sun casts a seven foot shadow in December, so parts of my beds are not good for winter crops. I used photoshop to figure that out, but if you have graph paper, a sheet of tracing paper, a protractor and a ruler, you could get a good idea of where the shadows are.

Hopefully, this is not too complicated. It's easier than it sounds:

1. Google "azamuth tables". I think the sight was associated with NASA, but I don't remember now. You type in your zipcode, and you can get the angle of the sun for your area at whatever day you specify.

2. Then get some graph paper and a piece of tracing paper. On the graph paper, draw a little scale model of your wall and the ground, using a scale of say, 1" equals 1 real foot. Trace off the ground line on the tracing paper.

3. On the tracing paper, use a protractor to draw in the angle of the sun, which is really the angle of the shadow your wall will cast. Match the ground line up with the graph paper ground line, then slide the tracing paper until the sun angle lines up with the top of the wall. Then all you do is count the little squares from the base of the wall to the point where the shadow line intersects the ground.

Susan

Please note, I can't respond to future comments, as I am writing this from the library.

I just joined Garden web so I could reply to this. My mother has problems with deer, rabbits, snakes, moles (or other mouse like critters) and raccoons. Each animal comes with its own challenges. Deer take bites out of lots of things and then they decide what they will eat to the ground. Raccoons dig things up to find what is underneath. We have finally found the answer. Old gazebo frames. My dad isn't convinced that is aesthetically pleasing, but it is working.

My sister and I set up the old metal frame (roof frame and all) on the side yard (adjacent to the forest preserve). We sewed heavy duty deer x on all four walls and let it extend onto the floor about a foot. Covering the roof is unnecessary for deer, but you could for birds.

We used heavy duty landscape fabric to carpet the floor, covering the extra Deer x, then we used U shaped metal things to hammer the landscape fabric and the deer x into the ground. This formed a bunny barrier. They can't dig under it. (they chewed through it once, but we just patched it with a square of deer x and plastic zip ties. On one of the four walls, though, we needed to make a door. We layed a wall sized square of deer x on the ground and attached it on the ground with the metal U's and then we bring it up like a drawbridge and just use hooks to keep it up.

Now she uses giant pots for her heirloom tomatoes, eggplant, herbs, etc. I keep telling her we could lay down some lumber all along the edges like a raised bed garden, but she is happy with her giant pots. (She uses plastic party tubs -the colorful kind with the rope handles that you would put a keg of beer in- they are pretty cheap. Then my dad drilled holes in the bottom for drainage.)

Now, as far as a potager goes, the deer x and the gazebo frame are black, so the whole thing sort of disappears (depending on the background), and you could add white picket fencing or lattice around the gazebo for the potager effect. Also, you can buy strings of large Christmas lights and tie them on the roof of the gazebo in an X pattern or a square. If you had a large yard, you could arrange 4 gazebos in a square with pea gravel in between and all around. I really don't know how they do it on Martha Stewart and the victory garden. If that was at my parents house, it would all be eaten.

I have been itching to share my secret. I bet you can get free gazebo frames on Craigslist because when the cover and the screen rips, it costs the same amount as a new one to replace the fabric.

Good Luck!

Joelle

As much as I love the idea of high brick walls for a potager, I think 6' tall walls would look funny at that scale. Basically, the height of the walls would be almost a third the width of the garden. I'm afraid it would look like a giant brick box sitting on your lawn. And besides, brickwork is expensive... VERY expensive. I'm not even sure that you could build a freestanding wall that high without bracing it somehow, either by making the base several courses thick or some other mason's trick. I doubt a mason would build a single course that high.

To my taste (and wallet), I think you'd be better off with a 2-3' high fence or wall of your choice immediately around the potager and then a separate plastic deer net around the rest of the property. My sister had big big problems with deer - they destroyed every living thing on her property, until she bought deer netting. If you're not familiar with it, it is thin plastic stuff that you can staple to a tree or post. A 100 foot roll of it costs less than $20. Search for "Deer netting on Google." It's so thin that from a few feet away you can hardly even notice it. Flimsy though it may be, it works! She encircled her entire 2 acres of property with it and now she has no deer problems. Actually, there was one more step she had to take, and that was to put a gate at the end of the driveway. The netting secured most of the border, but they started walking right up the driveway. She bought one of those fancy electric gates and secured it to couple of masonry columns. It looks nice, but I joke around with her that she must think her last name is Rockefeller because it makes the house look like an "estate." It's really just to keep the deer out though... until they learn the combination to the mag-lock! :-)

-Diggity