You don't have to remove the pups but it will be crowded. The pups are easy to remove, just take the group out of the pot and root around to find the attachment points and break them off the mother. I had a Guzmania that somebody gave me and it had 9 pups. That was way too many to just leave in a cluster. Now they are scattered around the yard mostly tied to bushes.

How do you just 'break them off'? Will they snap off easily? I live in zone 5a so putting them outside is out of the question. These are indoor plants and I'm worried that a mother and 4 pups is too crowded to allow 4 pups to bloom.

For now leave them alone...when it warms up where you are consider repotting the whole group...They can be indoor plants or shade plants in the tropical climates

It will be obvious how to remove them once you get a good look. In my limited experience with Guzmanias I find that they separate quite easily. When to do it is another matter. The general rule is 1/3 the size of the mother. But certainly when the mother is dead and dying it's time. They will grow faster if they have more space, and thus bloom quicker, but you may have to wait a year or two or three. I think it would be hard to grow one to maturity indoors unless you put a lot of effort into it.

To remove a pup I find where it joins mum, put my thumb there, grab the rest of the pup in my fingers and pull down. Some I have grabbed pup in one hand and mum in the other to pull apart. Somepeople do use a knife. If by chance you pull it off and it has not much stem on the bottom fill a small pot with potting mixture to where the bottom of the brom will touch it,[the leaves overflowing over the pot] then put it aside for a while and it should take root. Just treat it the same as your other broms.