Portia Blake, who is almost 11 years old, and her six-and-a-half-year-old brother Foster travel by train to spend the summer with their 12-year-old cousin Julian Jarman and his parents, Uncle Jake and Aunt Hilda, in the countryside near the villages of Attica and Creston. Their parents are traveling in Europe until August, making this the children's first solo journey. Julian, despite his ungainly appearance, is captain of his school's baseball and football teams and Portia's favorite companion, passionate about nature and generous in sharing his knowledge of birds, stones, and insects.
The next morning, Portia and Julian explore the woods. On a large moss-covered boulder, they discover a carved inscription: "LAPIS PHILOSOPHORUM / TARQUIN ET PINDAR / 15 JULY 1891." Julian recognizes "lapis philosophorum" as the Latin term for the philosopher's stone, a legendary substance alchemists believed could turn base metals into gold, but neither child can identify the names. Continuing over a ridge and into a vast swamp, Julian chases and catches a rare butterfly, then trips over an old rowboat buried in weeds. From its hull, they see roughly a dozen large, wrecked houses at the swamp's edge, adorned with sagging balconies, turrets, and broken trim. Julian declares it a ghost town. Portia begs to leave, but from the last house comes the enormous sound of a radio.
They approach and knock. Minnehaha Cheever, a small, thin elderly woman in an elaborate old-fashioned dress with leg-of-mutton sleeves and a high lace collar, is thrilled to see children for the first time in years. She explains that the place was once Tarrigo Lake, a prosperous summer colony of 12 families, but the lake dwindled after a dam was built at Corinth in 1903 and vanished entirely by 1906. The families abandoned their worthless houses. Mrs. Cheever returned years later, widowed and nearly penniless, with her brother Pindar Payton, who had also fallen on hard times. They found their family's former summer home still intact inside and each chose a separate house at opposite ends of the settlement. She warns of the Gulper, a treacherous quicksand-like spot in the swamp, and describes the Villa Caprice, the sealed-up house of the formidable Mrs. Brace-Gideon, now swallowed by woods. She also mentions Craneycrow, a small island in the center of the former lake containing a little old house.
Mrs. Cheever summons her brother by blowing a conch shell. Mr. Payton arrives: dapper, white-bearded, with sharp blue eyes. He immediately identifies Julian's butterfly by its Latin name, revealing himself as an amateur entomologist. As he walks the children toward a safe route home via an overgrown wagon road hidden behind hazel bushes, he advises them to leave a marker at the entrance and mentions his childhood friend Tarquin Tuckertown. Julian connects the name to the boulder inscription, and Mr. Payton confirms that he and Tarquin carved those words. On the walk home, Julian and Portia agree to keep the settlement, which they call Gone-Away, a secret.
Over the following days, they return repeatedly. Mr. Payton shows them his vegetable garden, goats, and beehives; Mrs. Cheever leads them through the bog to her garden of rare orchids and carnivorous plants. Over chocolate cake, Mr. Payton tells the story behind the inscription. As boys, he and Tarquin kept the boulder as a secret meeting place, but one summer Tarquin returned from boarding school snobbish and accompanied by a condescending friend. They claimed the boulder for their "Philosopher's Club" and barred younger boys. Humiliated, young Pindar's father taught him a sleight-of-hand trick: At the boulder, Pin and a friend pretended the rock had magically turned their implements to gold, fooling both older boys. When Tarquin realized the trick, he graciously proposed a truce and carved both their names on the stone. Mrs. Cheever then offers the children one of the old houses as a clubhouse. They settle on Bellemere, the former Tuckertown house, whose spacious attic has dormer windows and good views. They name their venture the Philosopher's Club and spend days furnishing the attic with salvaged tables, a red rug, curtains, and a mounted moose head.
Julian insists on maintaining the secret. By late July, the children call Mrs. Cheever and Mr. Payton "Aunt Minnehaha" and "Uncle Pin," but their family knows nothing. Foster, suspicious about where Portia and Julian disappear every day, confronts Julian, who deflects. Portia suggests including Foster, but Julian refuses. Foster, however, has been following them. He trails them to the hazel bushes marking the hidden path, finds the marker Julian left, and follows the route over the ridge. From the hilltop he sees the swamp, Craneycrow Island, and the old houses. Rather than approaching the settlement, he crosses the swamp to explore the island, reaching it just as a violent thunderstorm breaks. He shelters in the small stone house, frightened but gradually calming.
Heading back across the swamp, Foster steps into deep mud that pulls him down: the Gulper. Clinging to a clump of grass, he screams for help. Mr. Payton hears the cries, races out with a clothesline pole, and tows Foster free just in time. Foster is bathed and dressed in antique children's clothing, and the old couple learns that Portia and Julian never told anyone about Gone-Away. At home, Aunt Hilda is frantic. Just as she prepares to call the police, Mr. Payton's Franklin, a lofty antique automobile, arrives with Foster and Mrs. Cheever aboard. Introductions follow. Uncle Jake, grateful for Foster's rescue, offers Mr. Payton a puppy; he and Mrs. Cheever choose one named Tarrigo. Aunt Hilda gently scolds Portia and Julian for keeping their friends secret.
With the secret broken, Mr. Payton resolves to bridge the Gulper for safe passage to Craneycrow. Julian helps salvage building materials from Castle Castle, one of the settlement's ruined houses. Meanwhile, Portia befriends Lucy Lapham, the visiting cousin of Foster's constant playmate Davey Gayson. The club expands with new members. When Tom Parks, one of the new members, reports that people in the nearby town of Pork Ferry mock Mr. Payton, both Portia and Julian hotly defend him.
August proves idyllic. Portia and Foster's parents arrive and are welcomed into Gone-Away's world. Mrs. Cheever tells the girls about the "summer cats": Mrs. Brace-Gideon would adopt kittens each summer, then have them killed in September. Baby-Belle Tuckertown, Mrs. Cheever's bold tomboy childhood friend, rescued them each year, stealing them during one of Mrs. Brace-Gideon's concerts and rowing them to Craneycrow, where the children fed them daily until a local man found them good homes. The completed Gulper Bridge is christened with ginger ale, and the group crosses to explore the island together. Foster and Davey receive ownership of the cottage. As September approaches, the boys find a turtle on the island with "TARQUIN" and "1891" carved on its shell, marked over 60 years earlier. Mr. Payton marvels that the turtle is at least as old as he is.
With summer ending, Portia and Lucy stumble upon the Villa Caprice's overgrown gateposts. The massive house is smothered under Boston ivy, every door barred and window boarded. When a discordant piano crash sounds from inside, they flee in terror. Julian and Tom enter through a loose bay window and explore dusty rooms containing heavy furniture, a red-and-gold piano, a suit of armor on the staircase landing, and a chandelier. The "ghost" turns out to be a squirrel that had struck the piano keys. The next day, a large group returns to explore thoroughly. They discover a greenhouse, elegant rooms beneath the dust, and a round tower room with forget-me-not wallpaper that captivates Portia and Lucy. Aunt Hilda and Portia's mother speculate about making the house livable. Mr. Payton explains that under the Doctrine of Escheat, the unclaimed property reverted to the state after 50 years. Portia's father asks what the state might charge. When he poses the question to the children, Portia thinks of the tower room and pleads; Foster, thrilled by the greenhouse, the suit of armor, and the banisters, jumps with excitement. Their parents caution that obstacles remain, yet their mother is already choosing curtains. Mrs. Cheever, walking beside Portia, says she has a feeling in her bones that the wish will come true, and crosses her heart on the promise.