I'm scared to be alone with my horse....help??

ok,let me explain....i bought my horse Hershey(who is a QH/1-4TB to be two in june] in March of this year,and i've known him since his birth,he's always been kind of spirited and nibbly[which he doesn't do anymore thank goodness..lol.] but he's never been violent...well sometime in the ending of march/begining of apr. i took him out in our fields[which he knows] and he did not want to go,well i just figured he wanted his hay back in the barn so i kept going,but a few mins. later he kicked out at me[or trying to kick me] and then he bit my arm and starting half-rearing up,and striking out at me,which he hit me twice,and everytime he something to me,i smacked him[didn't want him to do it again so i tryed to show him he couldn't do it...]and he kept doing it for like 5-10 mins when i finally started to walk him back to the barn..by then[and i couldn't help but do this] i was crying,i was terrified and angry at him, i couldn't understand this horse that had just then attacked me....