This is because John and Elly are more or less loners by nature who really don't like being around other people all that much. This is why John needs to get wrecked so he can cope with the awful prospect of making conversation, it's why Elly is filled with existential dread when confronted with the soul-shattering terror of a child wanting attention and it's why they ended up living in a bunker with invisible walls. To one not afflicted with the need to flee his or her fellow human beings, Elly looks like a horrible parent who's too damned weak and fragile to cope with Lizzie's wanting a hug. Given that she's cursed with a social anxiety she won't treat or acknowledge, Lizzie looks to have a million or so hands trying to claw Mommy to bits.
The sad part of it all is they also have a need to conform to a norm that doesn't suit them. One of the stereotypes that limits John is the one where he thinks that he had to pair off and marry a wife who scares him and children who creep him the Hell out and Elly thought that she had to get a man and have her kids to be a real woman. Simply put, three children had to be born so they could be warped for life by two adults who would have been better off hiding away from scary, scary human contact miles away from the world and each other.