Also, it was my first time planting a bare root. I didn't even know which way to stick it in the soil! ;) I did however, have the "stalk" end up, sticking out of the dirt (in a pot until I see some progress) and the red buds are 2" below the dirt. Is this the way to plant them?

Is it a good idea to grow them in a pot until they mature, or should I go ahead and find a home for them in the yard?

White(ish), yellow(ish) or brown(ish) 'stalk' is a root. It should be placed flat on a ground with 'red (or white) buds' i.e. eyes pointing up. Then cover 'eyes' by 2" (not more, 1" is OK too) of soil. Make sure you have at least 7-8" (the more, the better) of good soil with plenty of organic matters below root.
In any event, either you decide to keep it in a pot or plant directly in ground, you have to replant yours the right way.

It's not necessary to grow peonies in a pot until maturity, however I'm myself rarely plant peonies directly into grounds and I think I have a good reason to do so.
In general, peonies don't like to be transplanted and sometime may not bloom for year or two after being moved. Moving peonies should be done in a fall only.
Contrary, potted peonies could be planted at any time of the year for as long as grounds are workable.
After being planted in proper location they could live and prosper for many-many years if left undisturbed.
So, one have to chose their location in a garden carefully with provisions for their future growth and for the growth of surrounding plants, shrubs and trees as well.
Therefore, unless I'm absolutely positive about their placement, I pot them up in a fall, grow for two years until I see the flowers (to make sure they are true to the name or to my liking), move pots as many times as I wish to and during the second season plant them permanently. Even though ocassionaly I have to move them again which result in at least one year setback.

Good luck.