This need to not face her face has its equal in John's not especially wanting to see that the things he says tell us things about him he doesn't want to know about himself. As josephusrex reminds us, John's entire career is built around the fact that he doesn't want to admit to being the insensitve clod, choleric petty tyrant, entitled clod and misogynistic tool that we know him to be. This means, of course, that he won't admit to himself that when he calls his employees 'his girls', wonders out lood what exactly Elly's problem is with being thought of as 'Mrs John Patterson', bleats that his children are too young (and Elly too old) to be so 'sensitive' or makes dismissive comments about April's feelings of loss and confusion as being the meaningless drama of a princess, he's outed himself as being a smug rat who doesn't care about the feelings of those around him because doing so is a sign of weakness.
That's right. I went there and I brought stuff back too. John doesn't want to admit that he's a jerk because he equates admitting error as a sign that he has no power. To him, freedom's just another word for being a selfish dunce.