Something I do not like

I am so rude however I would like to write what I think in my daily life here.
At my work, I moved to a new team and work with new people.
I am quite happy basically.
However there is a one man who I am not really happy with although he is nice.
He is a divorced man with a children who lives with his divorced wife.

He is nice however I can not have any affection to him.
I normally have affection to people even to people who are not popular among us though.
I still treat him nice since I think it is not fair since he is nice and do not harm me at least directly.

Why I dislike him? Disliking him makes me feel really bad. Am I a person who dislike people without reasonable reason?
Especially I do not like him when I find he looks at me when I do not notice and find it.
Please do not start in that way.
I am so small mind person.....however it is my real opinion..
I think I only need to communicate with him at work so I can just treat him nice in a usual way and I can ignore the feeling of disliking him.
I do not need to like him.