I can not think of a single nut tree that doesn't get huge.
Perhaps your best bet is to stick with the fruit trees and try and find a friend you can trade with at harvest time.

The one danger of nut trees is that they often replant themselves in places where you'd just as soon they didn't grow to begin with.
I've got black walnuts all over this property. Getting rid of a sapling that is growing where I don't want it can be a lesson and a half in patiences and determination.

Hi Carol, thanks so much for your input. I have sort of come to the same conclusion, in my own uneducated way. I think we will stick to the fruit trees for now.

Miller's Nursery in Canandaigua, NY carries Heartnut - the Japanese Walnut. According to them, it is a spreading tree that reaches 20 - 30' tall and 20/25' wide at maturity.
Fingerlakes Filbert and Chinese trees all seem small for your specifications.
Also check out Oikos Tree Crops for other nut trees, some may fit your size requirements.
You could google Small Nut Trees and see what you come up with.
Hoping this helps.