Is it so hard to ask for someone normal?

I don't understand why some people like to be funny and play games and stuff, I mean yeah it can be amusing but some people don't know the limits. I don't think it's an excuse that it's all subjective, especially when it comes to someone that knows you very well. If it's just a casual friend or something then it's ok, but if it's someone you know very well such as a lover or boyfriend and they know you don't like something and yet they intentionally do it, then I think that's wrong. Even if they wanted to joke about something or whatever, if they already know it's a touchy or very sensitive thing then they should not..so is that reason enough to avoid or minimise contact with the person if they continuously do so? If we feel unhappy about it should we not remove said source of unhappiness from our lives?