Wasps are a fairly large and diverse family that can be very aggressive or fairly gentle.
I would suspect those living in the ground would be Yellow Jackets and those can be very aggressive and at this time of year are looking for protein to eat and feed to those that will survive the winter as queens to establish new colonies next spring.
Just covering the entry/exit you can see is not going to do much of anything since they always have another and will dig out the one you covered again anyway. A large clear bottle can be placed on that entry/exit to trap them as the emerge, put it in place at night. Carbaryl dust placed so they must walk in it when entering the nest puts that poison down into the nest where it will kill off the queen as well as the workers, but shortly the weather will kill off this nest and wasps tend not to use the same nest year after year so maybe nothing needs to be done this year.

Here is a link that might be useful: Wasp images

Thanks.