We know, of course, that said constant failure to let children just have quiet, unstructured fun comes from John's need to have his children's unquestioning obedience (as well as to have them grovel pathetically for forgiveness every single freaking day for forcing him to spend money on them) and from Elly's need to step on relationships to avenge herself on all the pretty girls who did better at romance because they weren't slouching, frowning morons who wore dissatisfaction with their lot in life like a suit of armor.
The question we must ask ourselves is "Did their parents do anything remotely like that to them?" I could almost understand it if the Patterelders or the Richardses were in the habit of packing their children off to summer camp or something but I really don't think so. What we're dealing with is dumb, mean people making a hash out of what in the hands of competent, caring parents would be the favor of getting out of the city to see something new. All they see on Exile Farm is more adults griping about how lazy and spoiled they are.