We as humans do not value lives consistently. While we are willing to act for one victim, we often become numb as the number of victims increases. The empathic ability to adopt others’ perspectives is essential for motivating help. However, the perspective-taking ability in our brains seems limited. Using functional MRI, we demonstrated that the core empathy network including the medial prefrontal cortex (mPFC) was more engaged for events happening to a single person than those happening to many people, no matter whether the events were emotionally neutral or negative. In particular, the perspective-taking-related mPFC showed greater and more extended activations for events about one person than those about many people. The mPFC may be the neural marker of why we feel indifferent to the suffering of large numbers of people in humanitarian disasters.