Maybe wear a hairnet, like kitchen workers do? Maybe do your gardening closer to evening? It's not impossible that their guard bees are telling you to back off.

Look at moving the hives (first choice) or at least facing them a different direction. Yes, hairspray, perfume, lotion, etc. can attract them and if you start swatting at them it'll just make the situation worse. Most likely at this time of the year, colonies just get naturally more "defensive" of their nest area because it's loaded down with the food supplies (honey) they need to survive the winter - this becomes a prime target of opportunity for other animal and insect raiders. Thus the bees have the natural instinct to fend off any potential robbers in a more aggressive manner.