Look at the "My New Rock Garden (Photos)" thread that doggonegardener posted. You'll get an idea of the process. :-)

Putting a rock garden on flat ground under a large tree is not the way to go in my opinion. The tree is out of scale and because of the shade and root competition, makes it very difficult to raise plants. If grass can't grow there why plant sunloving rock garden plants?

I have found out that poor soil works better for succulents than good soil. If I was going to make another rock garden on a slope, I'd start out with 5/8's minus gravel and the biggest possible rocks I could manage and then dig pockets for the plants. I'd have less trouble with the weeds. The rocks concentrate the water. The rocks should all come from the same formation and quarry. A haphazard collection of rocks from all over doesn't do it in my opinion. It's gotta look a bit natural, not contrived.

We all have our own taste as to what looks good and what doesn't. There are some really outstanding rock gardens on the gardenweb. Krensgarden is made to look like a quarry pond and has mostly the same rock texture. their garden is on flat and hilly ground. Our rock gardens are on flat ground and made from every size and type and shape rock imaginable. Plus we have a grove of trees and we surrounded the grove with several individual sections, all tied together, incorporating the outside ring of trees. Each section has it's own theme. One has a fox sitting on gravel staring out over a piece of driftwood with huge flat boulders behind it and azalia bushes behind them. We also blended in railroad ties, whiskey barrels and several other statues and a fountain. It may not look natural but it's a thing of beauty. In this area of the Ozarks you will find rocks of various structure and material side by side. You will also find many of them have fossils in them. It is one of the latest structures of earth that erupted from the sea .

Yes, I agree, it's a matter of taste. A bunch of random rocks that look like they were dropped from a helicopter, placed helter skelter with a large assortment of plants placed the same, amongst the rocks, is not good design. Add railroad ties, whiskey barrels, and an assortment of 'garden junk' to the mix and it's looking like a Phunny Pharm. Good taste? I think not. Certainly not good design.
C'mon, let's raise the bar.

Here is a link that might be useful: Part of one of my rock gardens

There are many good books or sections of books available. Each has their own vision and here's mine. As stated, The larger the rocks the better and all of similar type. Setting them in tilted or canted layers also tilting just slightly back into the landscape, to catch water and helping keep the pockets of soil from washing away. They do not seem to look as good when on flat land, but consider the burran area in Ireland. Low, ground hugging plants dominating and in groups of 3,5,7 or more and irregularly distanced from each other. Obviously, I like the natural look.

I am not an expert but this is what I've done;
I have a rock Garden in a very small area with not enough sun to plant anything, well maby moss!
Where I started was by digging up the patch of grass, the moss and mounds of dirt. I then leveled the ground, put landscaping fabrick, then 1"washed rock as my main landcape. I put a walk way From the gate to the door steps with Edmonton brown stons that are naturally 1/4" brown and round. To traverse this walk way I placed really large(100-150lb) flagstones.
I added a spot for an above ground moveable fire pit. The resting spot is approximatly 4.5' in diameter, and consists of 1.1/4" Granulite/popcorn rock(burnt clay). When the fire place is not there(Canadian Winters) I have a 5' low rise table which is nice to sit and converse around of course the center piece is a couple of small flagstones and dark laquered driftwood.
Because of the lack of sun I used large exposed aggregate concrete planters to tie in the atmosphere. in those I planted Herbs and spices which really flourished.
There are piles strategicly placed of old pitted rock, orange, purple and brown, everything natural and beautiful.
I hope this description of my garden may be of use to you in creating yours.

There's lots of good advice here about creating a rock garden but that might not necessarily be what you want. Are you looking to create a relatively formal succulent bed? A pseudo-habitat succulent area? Or a rock garden as described here, but populated with succulents rather than alpines?

Your collection of sand a and gravel should make an excellent start for planting succulents. Mix it with your soil if you need to improve drainage a little. Think about a raised bed or a sloped area if your water table is high or the subsoil prevents you getting good drainage through the soil. Save some clean pea gravel for a top dressing if you want a fairly formal bed.