You see, a sociologist named Dunbar noticed something a while back when studying how people build social groups. It seems that for some reason or another, an average person can only build a stable social network with about one hundred and fifty people. The vast majority of mankind thus tends to become somewhat less human as anyone outside the monkeysphere devolves into either a piece of scenery that moves around or an appliance that performs a useful function. Being asked to care about anyone outside the friendship zone takes a certain amount of mental effort as it's being asked to care about a sofa or a paving stone.
That being said, it seems to me that the Pattersons are somewhat deviant in that respect owing to their own individual Dunbar's number being one. The solipsistic weirdos have to think that if they display empathy, it means that they're going to be deprived because they can only think about themselves. It's in their inhuman nature.