None of the conditions that you mention should be an attractant to wasps. Maybe the plants that you grow are attracting them. Look on them as the white hat brothers who will help you to keep away the bad guys. It is possible that you can work around their schedule! Also not all wasps are aggresive. Identify the wasp and go from there.

Most wasps, like bees, are not real problems unless you threaten them. Now the Yellow Jackets, also wasps, can be just mean and they should be a concern, but if what you have are not the Yellow Jackets there is no real cause for concern. I have the Wasps that build the paper nests in my yard tool shed and they never bother me as I am in and out of there just as the bees working the flowers for pollen never bother me as I work in the gardens. But I have been chased, and stung, by the Yellow Jackets for no reason I am aware of and those buggers I try to eliminate even though they do help control many insect pests. So you need to identify which wasps you have before determining whether some control is needed.

Depending upon the species, wasps may be collecting the building materials for their nests. Paper wasps will collect fibers from wood and other materials. I've had them gather fiber from the very book I was reading on the patio! You have to be careful about turning the pages, when that happens, lol. Others will collect soil (or mud) from the garden to build their type of nests. This would be the time of year for nest building, in your location.

I'd be observant for these nests. If you find any in busy locations, you may need to take some action, but other than that, I wouldn't worry too much about them.

Wasps are predators and they eat bugs mostly. Some hunt spiders; I've watched the action. If they are hanging in your garden they might be hunting the bugs that eat your plants! Paper wasps & mud daubers won't bother you--they are looking for wood fiber & mud, to make egg chambers to put their babies in---with loads of stunned bugs for baby food.
Yellow jackets (yellow & black) are cranky & get even more so in late summer/early autumn, when their food supply dwindles & they seem to know that most of them will die soon. They nest in the ground in former vole holes. You might run into a problem with them if they have put a nest in some vole hole in your garden. Watch for action in & out of a hole in the ground. If they are being aggressive to you, or the nest is in a spot where you have to walk around, work etc, wait till dark & spray one of those wasp killer products in the hole, liberally.
In late summer yellow jackets can be a pain at your BBQ, trying to get a hunk of your hamburger right out of your hand. Put bait some distance away from the food area, before the party. You can get yellow jacket traps--just google them.