Terrace your hill. Just put a row of logs, railroad ties or 6 x 6 redwood posts secured with rebar along the edge. Then fill in about 2 feet of flat soil, and repeat the process. You should end up with a stair step (steps being 2-3' of flat soil wide) look. Water will settle in and help the plants instead of washing down the hill.

dhaval,
Look at the 2 pics where 2 places on your hill the water is pouring down. There are specifically 2 places where almost all of the water is running down your hill.
Give the water another way to run.
Go to where the water starts to enter , at the top of the hill in both pics, and dig a trench lower than the ground level around it. The water will run to the lowest ground.
Dig it along the back of the hill, at a slant, running down to the bottom. In one pic this will be by your fence. I think to the left. In the other pic, do the same thing.
ALso, find a place close to where you live that sells rocks. They are cheap, I can buy alot of them for 15.00.
Use the rocks also for around the trench you dig in.
Then, look at Home Depo for the cypress mulch they sell for hills. It is shredded. It is specifically made NOT to move during rain storms on hills.
Apply this with gloves though, believe me, it works.
It is a light colored mulch, you can throw your pine straw on top of it if you want if you don't like the color.
It stays where you put it. It's a god send for me.
Trench, rocks. then mulch. You can throw some of the shredded mulch in the trench if you want, if not, just leave it.
Good luck.
Please let us know how it turned out later this year.
Noone ever comes back to let us know what the garden looks like afterwards.

you can try a geomembrane product to protect soil structures and foundations from water. i think that is help your problem

Here is a link that might be useful: Geomembrane Product