You didn't do anything "wrong" -- they probably nested nearby. Once the babies fledged and left the nest, they probably moved on with them to a different habitat -- the like the margins of woods and field where there is a diversity of foods for the young, insects, berries, etc.

I don't know about orioles, but many birds nest in isolation in a set territory, but then flock up later in the summer. I've seen large flocks of bluebirds, robins, and some other species in August and Sept.

You might see them again at the end of summer before they migrate. Typically, though, they do change their feeding habits around this time. You will likely see other birds hitting the fruit & jelly now though.