First off, it's somewhat ridiculous that the loutish imbeciles cluck their stupid tongues and wail like idiots about how some lucky one-hit wonder threw fun and friendship in the trash to follow some sort of evil, distracting star that deprived life of the real joy of what looks to most people like "standing around like an idiot catering to the whim of some dough-head named Patterson." People have memories and common sense and tend to remember inconvenient facts like how the Patterson girl is the one who turned her back on friendship because she's jealous of Rebecca-with-a-H because the blonde girl makes her feel insecure.
They also remember that most of the reason that the Pattersons hate the waster son's mother-in-law is not just because she tells the pinhead churning out abuse porn for Lifetime what he can do with his smug attitude and tendency to treat Deanna like his servant. What seems to be the real problem is that Elly resents having someone else barging on in and barking orders when that's her job. The idea that she's not loved best enrages and outrages her even if she didn't really do anything to earn it.
Most telling of all is their irrational hatred of Anthony's ill-used ex-wife. They see a cold-hearted, treacherous monster who's jealous of an intrusive idiot who won't go away for no reason. The people of Milborough see someone who, while probably not especially sympathetic, is to be pitied because she ran afoul of a group of clods who resent her for not admitting that her part in life is to not only disappear from Anthony's life because a Patterson is available for all his being a churlish dick to his spouse needs but apologize for being an obstacle to the imperial will of a Pattersaint.