Once returned to the prison, Hester becomes highly agitated. Fearful that she might hurt herself or her baby, a jailer decides to summon a doctor. The doctor is the person whom Hester noticed among other people in the crowd, named Roger Chillingworth. As soon as he enters Hester's cell, he insists on staying alone with Hester, making both Hester and the baby become quiet instantly. He prepares a potion for the baby, but Hester refuses to give it to her child, dreadful that Roger wants to revenge to Hester. He assures her, coldly though, that the child is not to blame for anything. After giving the child a medicine, he hands a medicine to Hester, when she admits that she was hoping to die, trying to investigate if he is now trying to poison her. Roger replies that even if he wanted a revenge, he would make sure that she is well and good, so that burning shame can blaze on her bosom. Hester drinks the potion in one gulp and they start a conversation about the ungraceful event. Roger admits that he should have expected something like this to happen when he married her, as she is young and beautiful, unlike him, who is old and ugly. He was wrong to believe that his intellectual gifts could be a substitute for youth and handsomeness. Hester replies that he knows she has never loved him, and he agrees, wanting to know who is the father of her baby. She refuses to reveal his name, like she refused to do so in front of townsmen and city officials, so Roger does not insist, assuring her that he will find out who is that man, this way or the other, but not to worry, as he has no intention of hurting or disgracing him. Furthermore, Roger asks Hester to keep his identity in secret, as devotedly as she keeps in secret the name of her child's father, so that he can live on peacefully, without being recognized as the former husband of the adulteress.

     In the following chapter, Hester leaves the prison. Although she is now a free woman, a letter on her chest will mark her for the rest of her life. Trying to start a life from scratch, she chooses to move into a house built by a settler who eventually left the property, as the land was not suitable for farming. The house is somewhat distant from the center of social gathering, therefore Hester and her child can lead a normal and peaceful life. Although she can leave Boston and go to Europe, Hester chooses to stay right at the "crime scene," as if she believes that she deserves to be punished.

     Although lonely and friendless, Hester is strong enough not to feel desperate. Her child gives her a strength to fight and move on with her life. Needle-work she starts in order to make a living, soon turns into a fashion, and she has more and more work. Her embroidery adorns clergymen robes, children's clothing, as well as the clothes of deceased, but it is not probable that her work decorates brides' gowns. She beautifies the entire town, especially her daughter, but never herself. All her clothes is plain and ugly, making the letter "A" on her chest even more noticeable. Whenever she walks the street, people stare at her, some of them curious, some of them fearful for the made-up stories about her being a witch.

     Chapter 6 introduces Hester's daughter. She named her Pearl because she represents the only treasure Hester has. As the little girl was conceived in sin, Hester dreads that God will punish Pearl as the consequence of the sin, so she keeps searching for some physical or mental defect in girl. However, Pearl is growing into a perfect child with extraordinary intelligence, although Hester starts noticing Pearl's variety in traits, as if there were several children in only one child. It seems that everything about this child is perfect, yet in disorder, making Hester ask herself if her child is from this planet. With a peculiar look in her eyes that puzzles Hester, it seems that the girl understands everything, as if she is much older than her age.

     However, there are some troubling aspects of Pearl's personality that terrify Hester. On some occasions, Pearl is obsessed with the scarlet letter on her mother's chests, so much that Hester has an urge to hide herself, while on the others, Pearl ignores its existence, as if there has never been anything on Hester's chests. Furthermore, one day when Pearl gathered a handful of flowers, she started flinging them at her mother's bosom. Flowers kept hitting the letter on Hester's chests while Pearl smiled devilishly. Noticing that peculiar look on Pearl's face, Hester asked her what was she, as if she was hoping that the child knew the secret ready to be revealed. But Pearl replied that she was her little girl. With a smile and shudder, Hester recalled the neighbors' claims that Pearl is a demon offspring.