Probably the main thing is if they have enough room on the other side, where they will not be diverted by a solid structure. Hard to cut these back without spoiling the shape, if not undesirable simply because it prevents them from growing larger and more impressive. They will get much bigger than 4 ft. across eventually, but it could take a long time. Ripping them out in 15 years would be a shame because of them having become fairly nice specimens by then. Some people are actually paying tens of thousands of dollars for roughly 70-90-year-old laceleaf maples (and these are an older, less attractive cultivar than 'Garnet') being dug out of properties being renovated (or simply occupied by people oblivious to the charm and significance of the trees) here.