What most of them see is a rather stubborn man who, having to contend with rather conservative parents who can't stop pushing the settle down and give us grandkids to pay us back for raising you agenda, tends to push back by delaying his transformation from hip young kid to boring old grouch as long as possible. He'd thus be seen as another poor slob taking longer than he really should to grow up because he acts like a small child faced with liver for dinner.
The ones that really have him pegged, however, are the ones who remember his crazy-ass older sister who seems to think that she gets to boss him around and meddle in his life because she has seniority. These unhappy few who are aware of Elly and her slobbering aggression take a certain amount of pity on the poor slob who fears living in a planet where he isn't allowed to be anyone else but the boy who broke her damned doll. They assume that when Elly finally grows up, so will Phil.