Remove old lattice
The brick bed is very small to plant most conifers. You could try Ilex crenta Sky Pencil - but it will take time from them to grow.
I would extend the top of the fence , maybe not using lattice like in the photos - but 2x4' and allow a vine to grow along the top ( if you need plant material.

Here is a link that might be useful: extend top of fence

Plantman56! I've never seen Sky Pencil before..they might do the trick, Thanks! My neighbor put up the lattice last year on his side. I think adding some on our side using the steps outlined in that great link you put up might be nice.

Here is a Pic looking towards the back. You can see where they just added lattice with no frames. I'm hoping to find something of height to wrap the whole yard with and get privacy.

Here is a pic looking to the right from the back porch.

Is it your fence or his fence? If it is yours, well you have right to remove or alter With the second photo, there appears to be a little more room. There are a few tall narrow yews that will work ( if it is shaded) The tree on the right, will have roots that will make it tough to plant.

blue arrow juniper?

Here is a link that might be useful: blue arrow juniper

plantman56! - It pre dates the both of us but I think its his. With how close our houses are we have been lucky in that we have a great relationship built up between us and what ever we decide to do they'd be completely fine with. Anything I put up on the fence I'll probably have to make work from our side. BTW that one section of fencing facing back into the yard is just a spare I put up there temporarily to keep straight for use later as part of a gate.
mmajicmann - blue arrow juniper is also new to me, thanks for the link. Reading through it sounds like they hold up well in urban environs and do okay in Zone 5.

There are a couple of upright, narrow types of boxwood that would fit that spot nice. They will tolerate shade and are deer resistant (read: nothing is deer proof).

- Buxus sempervirens 'Graham Blandy'
- Buxus sempervirens 'Dee Runk'
- Buxus sempervirens 'Fastigiata'
- Buxus sempervirens 'Green Tower'

In a few places for extra privacy... I would add some tall trellises... then add climbing roses or something at the bottom. Once it fills the entire trellis, you'll have solid privacy. I purchased some climbing roses and within a year they are nearly 10' tall. They still need to fill in more, but they grew straight up. Along with that, Add something like the picture below... then plant something on each side to climb up and over. By the way, nothing as fancy as this, since the vines will cover most of it.

In summery: instead of putting up additional wood to look at, I would add structures that you can have stuff climb. Then in spots that you have more room, add a few narrow plants that grow straight up

And the trellis

Sorry for the delayed response, thank you for the input!

and what did you decide.. how about a followup pic

ken