There is a part of the brain known as the sensory cortex. This part of the brain deciphers sensation, so any time you are touched or touching something, the sensory cortex is what gives you this information. The thing is, the sensory cortex doesn't spend its resources equally across the body. There are some parts of the body that it pays attention to more and some parts it pays attention to less.
Take a look at a [homunculus](https://upload.wikimedia.org/wikipedia/commons/thumb/4/42/Side-black.gif/800px-Side-black.gif
). This is a representation of how the sensory cortex divides its resources across different parts of the body. In the homonculus, the larger the part of the body, the stronger the sensation in that area. You can see that the hands and face are absolutely massive meaning the brain pays a lot of attention to those parts of the body. In reality, your hands are the most sensitive part of your body because it picks up more information. Your face is a close 2nd.
The skin on the face is more sensitive than skin on other parts of the body because the layers of facial skin are thinner, and there is a greater number of nerves. The nerves send the signal of touch to the brain with much more detail, so we can sense when our face is being touched very easily, important for immune response.