Hmmm. Given that he isn't what you'd call welcome when he does go to the library, my guess is that it just might have done pea-brained Elly in to have to figure out what to do with him. Since she doesn't want to KNOW that he's wandering the streets (while otherwise being fine with that when she doesn't have to think about what he's doing) and since she doesn't want to be downgraded to mere worthless housewife by having him show up at her big important non-job and take her adulthood away from her, it's either have him suffer there or have him suffer by being lectured about being a selfish monster who hates her.
What Elly doesn't seem to want to realize is that by incessantly lecturing her children about how being nothing more than their mother is a fate worse than death, she's setting up a scenario in which they don't actually care much when she does lose her job. While not averse to the idea of Elly working per se, what they do resent is being told that they're evil parasites for wanting attention and hateful invaders for wanting to go where Mommy works. More on that moronic idea tomorrow.