Wet soil is in my experience the best aid for pulling weeds, roots and all.

The answer to controlling them in the future is not weed killer (after all, this is toxic to all plants, except for Monsanto's GM crops), but rather mulch. If you cover all bare soil with a thick layer of mulch the seeds of those annual weeds have no chance, and your shrubs and trees will be the happier for it as well.

I mulch often with barley straw because where I live it is plentiful, cheap, and easy to transport, store, and apply. It really controls the weeds while maintaining moisture and adding organic material. You might find that rice straw from the central valley is similarly useful.

But if your garden is a more formal space (mine is strictly for growing food), then your choice of mulch will probably depend on aesthetic qualities as well as availability. Some people use a layer of cheaper mulch like straw or leaves (or of course breathable landscape fabric) overtopped with something more decorative, such as bark, wood chips, pecan shells, stone, etc...