ground bees or yellow jackets or something else?

I live in a suburb of Kansas City but there is a wooded area down a hill behind my house. I made the mistake of allowing the grass to get somewhat overgrown over the past 2 weeks because I've been working a lot of overtime. So I was cutting the grass in my back yard this morning and came across a nest of, not sure what. They did not seem to be aggressive. I was very careful when I spotted it but continued to cut around the hole in the ground. They came and went without aggression. When it came time to cut over the top of the hole I backed off for a few minutes and finished the rest of the yard. Then I quickly ran the mower over the hole and left without incident. The back yard is very small and the are no trees or bushes. It is enclosed with a 6' wood privacy fence. I also have 2 dogs, 1 large and 1 medium, but they only go out to the back yard briefly several times throughout the day and have not been stung as far as I can tell.

Should I be concerned about these, whatever they are? And can anyone tell me what they are? They are rather small. They seem to be smaller than honey bees and they looked like they were, roughly, the color of bees but I didn't hang around much to find out. 2 things I am phobic about, stinging things and spiders  lol.

Artist,  To be afraid of something that buzzes and has a stinger is normal fight or flight behavior.  However, many of these critters are helpful or necessary for for many crops to produce their bounty.  Slowly over time, you should be able to relax more and more, even thanking them for pollinating the cucumbers, melons and squash, even tomatoes, peppers, etc. benefit from their buzzing in releasing the pollen.  Many live solitary lives except for mating.  Some have small colonies.  Some live in hollows of wood, some make "paper" nests, some in ground burrows, even in our SFG beds.

Some members have slowly overcome their fear and now plant "bee" friendly flowers.  A couple I know raise bees.  Another makes homes for mason bees.

I used to be terrified of big, black bumble bees and would run inside, sending husband back out with a can of Raid Wasp and Hornet spray.  Now, I watch them buzzing from flower to flower while working on plants nearby.  

Time and slowly acclimating to their behavior will help.  Now, on spiders, you're on your own.

They sound like standard ground bees.  They are small, not aggressive.  Let them be (lol!).  Yellow jackets are aggressive and will hunt you down!  If it is a pollinator, and peaceful and in a not high traffic location, I let it be! 

Welcome!

Thank you Sanderson and Scorpio Rising. I was hoping they were just ground bees. I don't like to kill anything, especially pollinators. I'm fine with bumblebees, honey bees and most buzzing things as long as they aren't aggressive. And even with my phobia of stinging things I've never had problems with honey bees and bumblebees, even carpenter bees. But wasps, hornets and yellow jackets with their aggressiveness make me want to stay as far away as I can get from them. I've only killed the ones that got in my house and only because I couldn't get them to go out safely. I've even left a door open and waited for them to find their way out. Which is much less daunting than trying to kill one. I know they all have important jobs to do for Mother Earth and, in turn, us. Even spiders, as long as they stay where I can't see them, are okay. But I manage to keep them out of the house, for the most part, (or out of sight) by spraying peppermint oil mixed with water along the windows and doors rather than poison because they are important too. I hate poisons and refuse to use them. My mom recently passed away due to lymphoma cancer because of, I believe, exposure to pesticides. It was the most brutal way to have to die. Nobody should have to suffer that way. And that includes the so-called pests that are vital to our ecology/environment.

I tend to have a lot of bug phobias, spiders being the big one, but I've become more comfortable with honeybees and bumblebees from my experience with them in my yard. I have yet to have either of them act aggressively. As you mentioned, wasps, hornets and yellow jackets, are a different story. Around here, especially yellow jackets. And I have had yellow jackets build nests in the ground on two occasions in my yard. I think you would know though, if they were yellow jackets, because of their aggressive nature. As long as you know where their nest is, and can give them some space, it sounds like you can co-exist peacefully.