I would just keep them going, but definitely clip the flowers. I put out some super chilis once that had already set fruit, and it took a long time for them to really start producing their main crop. I think once they have fruit, they prioritize where their energy goes, and work on the one or two fruits rather than growing roots and setting out more leaves. This seems to slow everything down.

If the plants are too small and they do form fruit, it will take them a long time (if at all) to catch up to the others.

But, chances are, the blooms will drop on their own.

Since you say they are too big already, maybe leaving the blooms on will slow them down a bit for now, and you can worry about whether or not to pluck the blooms later when it's closer to time to plant them outside.

Ok, I will clip the buds when the get a tad bigger. They are not even showing any flower yet, just the bud. I want to ensure they are actually going to be flowers and not leaf buds but I am pretty sure they are flowers. Thanks guys.