Francois..you're planning on planting in the ground? Year round? The Yuzu isn't hardy in your area during winter months; your plant will freeze.
You want to keep your Yuzu the same height for 4 more yrs? Is that what you're saying? Keep it underpotted for the time being. I know ppl who grow citrus as bonsai..they cut roots and keep in small containers.
For the time being, you want to know why your leaves are paling..Have you checked for spider mites? They can be difficult spotting, so you really have to do a thorough inspection locating them..you'll notice fine webbing in-between leaves. The insects themselves are hard to see..They're quite tiny.
Take plant to sink and spray..Don't keep soil wet at all times..buts like dry air and wet soil.
In summer, place outside. Fresh air does wonders.
Fertilizing is important..especially using a citrus fertilizer that contains everything including minerals.
W/O a pic of the leaves it's difficult ID'ing your problem..can you post one? Toni