In their farm house in a New York village, Ma Shelby prepares breakfast for her four children, Isaac, Tommy, Johnny and Susan, and then awakens them. The racket the boys make as they play and fight awakens their father, who spanks the eldest, Isaac. When a visitor chides Pa for not working, Ma sticks up for her husband, saying that he has a weak back and that he is waiting for a promised government job.

Jim Gladden, a construction site foreman, is partially responsible for the accidental death of one of his workers, Fred Smith, and makes good on Fred's deathbed request to go to Scotch Valley and take care of his surviving wife and children. When Jim arrives in the small town, he is told that there are two Fred Smith families in Scotch Valley, the rich Smiths and the poor Smiths. Jim assumes that the Smiths he is looking for are the poor ones, and is directed to a house where four children live in poverty.