Least..
Cool day's, early morning & late evening for honey bees, bumble's still fly in cooler temperatures but they are not harmful.
Wasp fly in cooler weather too, hopefully you don't have a nest near you.
Most..
Sunny warm hot day's.
In the garden when bees are left alone you should be fine.
Konrad

I am allergic as well and have an active beehive that adopted me last year. It has split off with a queen once that was sighted. With all this activity I have yet to get one upset enough to sting me ( bee sacrifice their life and die). But I garden like 'RAMBO' and I understand that when bee's are busy, they usually give a warning letting you know they are there.
The one impossible area for me to garden in is the HERB garden. Forget about it! The bee's are nuts for rosemary,oregano and the fruit blossoms from trees. I bought a head flashlight ( Home Improvment store) and can safely do a quick weeding while the sun goes down and they call are calling it a day. Always wear gloves and if your not sure, just take a garden stake and give the plants a light tap,and the honey bee will emerge if they are still present. Just keep in mind, the bee lets you know that he wants you out of his way by buzzing close by. They usually don't go further then that. Having a hive I did have a drone that was kicked out of the hive or something. Drones don't have a job outside of the hive, so this drone would just fly about near the patio and dive bomb me! I would run and the dogs would bark, it was here at this site I learned it was a drone, no stinger, and it happens sometimes.

If it was a drone honeybee,they have no way of stinging. Their job outside the hive is to mate with virgin queens.
As long as bees are foraging and not in need to defend the hive ,you should be okay.

I think there are wasps that like to make nests in spots like under the kid's slide, in trash can handles ....aside from spraying, is there any way to discourage them?

A swarm landed on my house, I think. There was roofing activity next door. The bees got in through bathroom window cracks. I killed 5 or 6, because of my grandchild. The other bees made it to a connecting room, about 30 bees. They all died. The swarm left, all this within 2 hours. Is the swarm gone? We have never had a bee problem before.