Good job. And good luck.

The cold weather will definitely hold them back growth-wise, but, I have never had it affect the fruit size. Not that I've noticed, anyway. If there are blooms on the plants when you transplant them, consider removing them and let the plant settle in and that next larger flush will come sooner. I wonder if being rootbound like store-boughts can sometimes be has an effect on fruit size.

Your situation brings back memories. For years I grew my own peppers from seeds. One year, due to my schedule, I couldn't start my own. So, I went and bought about 15 plants at a local store.

Want to know how many peppers I picked that summer? ZERO! Amazing, but true. It was the only year that I bought peppers, and the only year I never had any pods.

What happened? To this day I blamed the source of those peppers. The summer wasn't too cool, I hadn't changed my routine in growing peppers, etc. I had tried a multiple of varieties too, so something just wasn't right from the start.

What I learned? I will never plant peppers other than what I grew:)

FWIW - Don't wait to buy them, just put them in containers.

I live in the Chicago area and never had luck with peppers of any kind (even sweet). The turning point for me was potting them in containers. The soil warms up much faster than even raised beds, and the pepper roots love warm soil.

I've never done it, my crowded lighting situation prevents me from getting peppers going too early so they are about ready at about the right time temperaturewise. But I remember the lady I worked for at a nursery telling me that people always rush planting the peppers and tomatoes which seemed foolish to her. She said the tomatoes will sulk until the heat comes and the peppers can become permanently stunted if exposed to too much cold soil for too long. So there's some backup to your thought. :)

The turning point for me was also putting mine in containers, did that last year, I had more fruits than I had ever had.

What stands out to me is that you've planted toms in the same bed, they are also in the nightshade family. I would not plant them there for at least three years.

all the best