In a nutshell, the fruit is not affected even if two varietes cross. The seed can be affected so if you grow seed of Brandywine for example, you could wind up with a regular leaf plant. Tomatoes normally cross at a relatively low rate of about 5%. Under some conditions though, this can be much higher.

Fusion

Brandywine tomatoes are a HUGELY popular heirloom but I've not grown it because of my short season. There are plenty of newer disease-resistant hybrid varieties. However, I think that it would be far more likely that diseases currently in your garden soil would be more of a threat to your Brandywine than any possible jeopardy your tomato plants would have for future gardens.

Steve

However, Zeedman did a fine job on GW's veggie forum recently explaining how it can and does take place. As notes Zeedman - -

"However, some tomatoes are very prone to crossing:
(1) Potato-leaf varieties
(2) Currant tomatoes
(3) Double blossoms of large tomatoes (usually some of the first of the season)
(4) Varieties whose blossoms have a protruding style/stigma"

Steve