That's because we have to remember one thing about Elly: her fear that she doesn't really have an identity of her own. As we've seen, most of what ails her is the fact that she perceives herself as being an appendage of a male relative instead of a person in her own right. When the children leave her for the day, the extra time is not filled with thoughts of getting to know and like the person in the mirror; the time is spent worrying that she's no longer necessary to anyone.
This fear of being obsolete is, as we've seen, coupled with the inability to do anything constructive to combat it. It's better for Elly to feel bad about being on her own than to do something scary like really think for herself instead of defining herself by what she isn't.