Our asparagus grows taller than my 6' hubby and the bushy parts of the ferns spread wider than the 2' width of the bed we have them planted in. Basically, they're a small tree once they've gotten to their full height, at least in our garden. If they grow that big in your climate, then their mass might rival that of the large tomato plants you had and didn't like. They don't get that big for everyone, though. The ferns really are lovely and they do turn a gorgeous golden yellow in the fall. Ours don't flop over because the stalks are quite thick and sturdy (they get woody when they fern out), so a fence would be purely decorative. They're bushy, but not unruly. After they go through the yellow stage and start to turn ugly brown, they fall over as one big wave, though (ours are in a 20' long double row). They can be trimmed away at ground level before that happens.

Thanks Macky - I really didnt know they could get that tall, I was thinking 4 ft maximum. I think they will just go on the side of the yard and I'll put something else in the center of the potager.

Mine get quite tall too and really spread out. I grew mine from seed too.
I planted mine to the back along the fenceline. They are so pretty there and serve as a backdrop for the rest of the garden. In the Fall, the seeds turn bright red and the fern-like branches become a gorgeous golden yellow color. Very pretty.

You can get the look you were seeking with the Asparagus by planting Fernleaf Dill or French Bronze Fennel in the center of your Potager.

~Annie