In the tense years before the Civil War, when heated debates about slavery resonated in the halls of Congress, when churches increasingly denounced its evils, when free blacks were beaten in northern cities, when thousands of courageous, determined people managed to free themselves and escape north, Cape May was deeply embroiled in this struggle for freedom.

In Cape May black and white leaders of Pennsylvania’s anti-slavery movement and Underground Railroad renewed and refreshed mind and body to continue their efforts. From its buildings they issued stinging denunciations of slavery. And in Cape May the most famous conductor of the Underground Railroad, Harriet Tubman, earned money to continue her freedom runs.

The center of this activity was the intersection of Lafayette and Franklin Streets. It was there that founders of the Pennsylvania Anti-Slavery Society gathered, and Underground Railroad leaders took in the ocean breezes.

Take a moment to walk back in time to 1852, as we saunter down to 645 Lafayette Street. When we near the intersection with Franklin Street, a light-skinned African American man, wearing wire-rimmed glasses, one eye seemingly injured, stands in front of a two-story clapboard house. He greets us and invites us on to his porch to chat.

Stephen Smith is a friendly, but reserved gentleman, and only his well-tailored clothes hint at his great wealth. There is no indication at all that he is a major figure in the Underground Railroad.

As we sit down, his gracious wife hurries inside for glasses of fresh lemonade. She too is heavily involved in the struggle against slavery.

From the porch, we hear faint strains of piano music coming from next door at the Banneker House, one of the country’s only summer resorts catering to black guests. In and out of its front door walk distinguished looking men and women, chatting. We pick out snippets of their conversations, the words “slave catchers” and “Fugitive Slave Act” catch our attention.

From Smith’s front porch we see across the street as the door of Cape Island Baptist Church suddenly opens. A white man with a bushy beard and mustache rushes down its steps. We recognize him as Joseph Leach, who will, within a couple of years, become editor of the new local newspaper, the Cape May Ocean Wave. He is anti-slavery, and often preaches at the church.

We also glimpse the well-known white Philadelphia Unitarian minister, William Furness hurrying down Lafayette Street. He and his brother James bring their families to spend summer days at cottages nearby. Furness’s pleasant, round-faced countenance belies his intense and all-consuming hatred of slavery. The fiery anti-slavery minister’s son, architect Frank Furness, later owned a house in Cape May and is credited with designing the Emlen Physick estate.

And coming along Franklin Street, from the direction of the large oceanfront hotels, we notice a young black woman, perhaps 30 years old. She seems to know Mr. Smith. Someone nearby calls out to her, greeting her as Harriet. She acknowledges with a wave, her hand clutching some bills, her earnings as a cook. It is money she intends to use to return to Maryland’s Eastern Shore to rescue her family, still enslaved there.

These are some of the people we would have met back before the country plunged into civil war. Frequent steamers left Philadelphia in the morning and arrived in Cape May by evening, bringing summer visitors eager to escape the city’s heat and disease. Many of these passengers were leaders of the struggle against slavery.

But they were not the only people visiting Cape Island, as the town was known before 1851. Three or four thousand people crowded its streets during the summers, many of them supporters of slavery.

Situated on the southernmost tip of New Jersey, Cape May was at the crossroads of North and South. Just 17 miles across Delaware Bay were the slave states of Maryland, Delaware and Virginia.

Southern planters, eager to take in the refreshing breezes, also stayed in the huge oceanfront hotels. Steamers docked regularly from Norfolk and Richmond, from Baltimore and Delaware, bringing to town hundreds of southerners, including many slaveholders, along with their slaves.

At times, tensions between these southern enslavers, northern abolitionists, and free blacks, employed at the oceanfront hotels, were so intense that fights broke out.

“We have had a great battle in a hotel between the black servants and the white gentleman, which has caused some bloody heads,” famed Swedish novelist, Fredrika Bremer, who spent a week and a half at the town’s Columbia Hotel in 1850, reported to her European readers. “The greatest share of blame falls upon a gentleman who owns slaves. He will be obliged to leave.”

“There have been two attempts at murder in another hotel,” Bremer went on. And the Baltimore Sun told the story of chairs, glasses and other objects hurled across the dining areas of the Atlantic Hotel, when a pro-slavery Baltimore man attacked a black waiter. A Philadelphia man came to the waiter’s aid, reported the paper.

But a few blocks up, along Lafayette Street, Pennsylvania’s leading black abolitionists found a calmer corner in which to relax, somewhat removed from the drunken brawls and racial taunts hurled in the gathering places of southern planters.

Interestingly, this area was soon also the religious heart of Cape May. It all happened around the years 1845 to 1846, when Stephen Smith decided to make his summer home on Cape Island.

The only known portrait of Stephen Smith by James Stidun, circa 1840

Stephen Smith was born a slave, bound to a lumber merchant in Columbia, Pennsylvania. Extremely skilled and intelligent, he learned the trade and was running the business before he bought his freedom.

He opened his own successful lumber and coal yard in Columbia, a key port along the Susquehanna River. Dun & Company, a credit reporting firm, estimated that by the 1860s Smith was worth over half a million dollars [Editor’s note: equivalent in purchasing power to $14,458,253.01 today]. Martin Delaney, writing in Frederick Douglass’s newspaper, North Star, declared Smith “a remarkable man in many respects,” and “decidedly the most wealthy colored man in the United States.”

Smith moved to Philadelphia in 1842, letting his business partner, William Whipper, run the company in Columbia, while he concentrated on real estate, buying and selling dozens of properties in Philadelphia as well as a handful of houses in Cape May.

With his riches, Smith could have lived a life of ease. But believing that no person of color was truly free as long as slavery existed, he devoted himself to ending it. He was involved in the abolitionist movement early on, helping to launch the Pennsylvania Anti-Slavery Society.

And he was a leader of other organizations demanding that blacks have the right to vote, and access to jobs and schools. He financed many charities, ultimately leaving the bulk of his fortune to operate the country’s first nursing home for blacks. Horrified to see the devastating impact wrought by the cheap whiskey shops inundating the black community, he and Whipper led many black abolitionists to form temperance groups.

Smith and Whipper also began work with the Underground Railroad in the 1830s. Their lumberyard and offices lay at the foot of the main bridge crossing the Susquehanna River, the demarcation between “the slave-holding empire,” said Whipper, and freedom. It was the first stop for many escaping slaves on their flight north.

They hid people in their homes, then sent them, by foot or wagon, to conductors further along the route north. But, as the number of people fleeing slavery increased and slave catchers prowled town in search of runaways, this became more dangerous.

Whipper came up with a clever solution. They shipped their lumber and coal in their 22 railroad cars. Why not use them to hide freedom seekers? So they built false ends to the rail cars, creating hidden compartments. Hundreds of people rode their way to safety along this actual iron and wood railroad. Inspectors on the railroad never once discovered them.

The freedom route wound about 80 miles from Columbia east to Philadelphia. As the train slowed down near the city’s Belmont Mansion, people jumped from the cars and were brought to William Still, the northeast coordinator of the Underground Railroad.

Banneker House

While Smith set about building his home in Cape May, his friend, James Harding, was expanding and renovating the building and structures next door, preparing to open what was perhaps the only summer resort for free blacks in the country.

Harding, a well-to-do undertaker, and Smith were both members in Philadelphia’s newly created black chapter of the Odd Fellows, an organization to help families who lost their breadwinners. The new group was the brainchild of a leading abolitionist and Underground Railroad leader, James McCrummell.

Harding named his hotel the Banneker House, after the renowned 18th century black inventor, astronomer, and anti-slavery activist. “Ho for the Capes,” Harding announced in Philadelphia newspapers in 1845. “The subscriber respectfully begs leave to inform his friends and the colored public generally, that he has opened a boarding house for their accommodation on Cape Island.”

The only known photograph of the Banneker House, seen left. From left to right: Emily Dempsey, Charles Bose Jr., Margaret Davis, Barbara Saunders. Circa 1940s.

Offering more than 30 beautifully furnished rooms, good food, and entertainment, Banneker House became a summer resort for what the Baltimore Sun termed “the colored aristocracy.”

Harding’s guests were small business owners, skilled craftsmen, professionals and artisans. They were devoted to educating and advancing the broader Philadelphia African American community, the largest population of free blacks in the north. They helped open schools, debating societies, libraries, and numerous forums for scientific and philosophical discussions.

Members of Philadelphia’s Banneker Institute, set up in the early 1850s, summered at the Cape May hotel, which shared the group’s name. The Institute sponsored lectures and debates on political, literary, scientific and social issues, especially slavery and equal rights. Its key organizer was Jacob C. White, Jr., whose father was a leader of the Philadelphia Vigilance Committee, which oversaw the Underground Railroad.

The group’s Cape May visitors included men like John C. Bowers, an activist in the American Anti-Slavery Society. He worked with Smith in the convention movement, which organized blacks throughout the north to fight against slavery and for the right to vote.

In 1855, calls for freedom rang out from the steps of Cape May’s Banneker House, drafted by Banneker Institute members. The group was marking the anniversary of slavery’s end in the West Indies by hosting a First of August Celebration. They issued a series of resolutions, signed by such men as J.C. Bowers.

“We lament that the United States, although boasting of her liberty and republicanism,” they declared, “still hold in slavery 3½ million of our brethren, thus demonstrating to the world its great inconsistency and injustice.” They called on blacks everywhere “to use every effort for the overthrow of slavery and for the acquisition of our rights in this, the land of our nativity.” Freedom’s spirit was rising, they declared, and the days “of chattel slavery are numbered…”

Printed in Frederick Douglass’s newspaper, these resolutions circulated in towns throughout the north.

The Religious Heart of Cape May

Stephen Smith was a deeply religious man, accepted at an early age as a local elder and deacon in the African Methodist Episcopal church. He opened churches in Columbia and elsewhere, and was an important leader at most AME national conventions.

So, it’s not surprising that when he came to town, the black members of Cape Island’s Methodist congregation decided it was time to form their own church. They resigned from the Methodist Church in 1845, and within days applied for recognition from the African Methodist Episcopal Church.

They wrote to Rev. Henry Davis, who oversaw South Jersey churches, and were granted their request. Rev. Davis preached in Cape May over the years, and was a fierce advocate for women’s equality. He led an effort within the church to allow women as AME preachers.

Cape Island’s Presbyterians, Baptists, and Methodists did not have their own buildings in the early 1840s, but met in the local school building, down Franklin Street, where the firehouse now stands.

In 1843 the Methodists decided it was time to have their own church and built one, where the Franklin Street School now stands. In 1854, the white congregation moved around the corner and sold that building to the AME congregation.

Throughout the north, the AME Church was devoted not only to bringing people to God, but to educating and supporting the black community. Many churches functioned as Underground Railroad stations too, although we don’t know that the Cape May church did.

George Howell, a wealthy Philadelphia businessman and Quaker, bought much of the land adjoining the original AME Church, beginning in the 1850s. Decades later he donated land and his home to the Macedonia Baptist Church, an African American congregation that now stands on the corner of Lafayette and Franklin Street, next to Franklin Street School. Howell’s house became its parsonage.

The former Cape Island Baptist Church, now condos, at Franklin and Lafayette. | Photo: Morgan Sacken

A year after Stephen Smith’s house went up, a white Baptist congregation completed their own building, Cape Island Baptist Church, directly across the street. This congregation was an offshoot of the First Baptist Church in Cape May Court House, a staunchly anti-slavery church.

In 1841 the Court House Baptists sent a resolution to the state Baptist association “expressing our abhorrence of the crime of slavery.” They declared slavery a “moral and political evil, a sin against God and a disgrace to the Nation…” Wanting other churches to take a stand against it, they warned it was “an evil that will never be done away while it continues to receive the countenance and support from Christians.”

The Cape Island church was organized in 1844, when some two dozen of its members living nearby decided they had a large enough community to set up their own meetinghouse closer to home. Their Lafayette Street building, which opened in 1847, was later used to sign up Union army recruits. A newer building was erected in 1879. Now known as the “yellow church,” it has been turned into condos.

One of the church’s most prominent members was Joseph Leach, a licensed preacher, and for decades a church trustee and deacon. He preached often in the building across from Stephen Smith’s home. Leach also helped establish the county Republican Party, to back Abraham Lincoln for president.

As the editor of the Cape May Ocean Wave, Leach covered the story of a group of people who fled slavery across the Delaware Bay. They had commandeered a government boat and landed near a community of free blacks at Fishing Creek in 1858. A free black man, Edward Turner, had extensive land holdings there. Turner later told a longtime Cape May educator that he had been part of the Underground Railroad, using his wagon to help people fleeing slavery.

Leach’s view of slavery became clear in his reporting. He not only knew many details about the freedom seekers, but also their escape route north. He either was involved or knew those who were. “We glory in the spunk of our ebony friends,” he wrote in the Cape May Ocean Wave. “We would do the same if placed in the like situation.”

Philadelphia blacks were not the only abolitionists who summered in Cape May. The city’s leading Quaker and Unitarian anti-slavery leaders were also seen on the city streets around 1850. They included family of Lucretia and James Mott, two Quakers who helped lead the Pennsylvania Anti-Slavery Society.

William Furness spent hours in Philadelphia with William Still, coordinator of the northeast Underground Railroad, listening to him interview people who fled slavery. “Among the abolitionists of Pennsylvania, no man stands higher than Dr. Furness,” William Still later wrote. As to the Underground Railroad, Still said Furness was “one of its chief co-laborers.”

Fredrika Bremer, who spent time with the Furness family in Cape May, declared William “one of the noblest, purest human beings whom God ever created.” She feared he was “so absorbed by his anti-slavery feelings that his life and his mind suffer in consequence. He would with the greatest pleasure suffer death if by that means slavery could be abolished.”

That nearly happened to one of Furness’s friends, Massachusetts Senator Charles Sumner. Sumner was beaten senseless on the Senate floor by a Congressman in 1856, after Sumner denounced two pro-slavery senators. The attack immediately escalated tensions between North and South, as thousands of northerners took to the streets in support of Sumner and a million copies of his anti-slavery speech circulated.

Sumner was taken to Furness’s Philadelphia home to recuperate, but he was not doing well. His physician, Furness’s son-in-law, suggested a trip to the seaside. So, that August, Charles Sumner spent several weeks at the Furness cottage in Cape May, in an effort to heal from the blows of the pro-slavery southerner.

Interestingly, Leach’s Cape May Ocean Wave did not mention Sumner’s presence in town, perhaps fearing for his safety. But in one of its few news articles that summer, it reported that his attacker, Congressman Brooks, had resigned his seat in Congress, then immediately was re-elected by his South Carolina district. “These proceedings argue strongly against the South and their peculiar institutions,” he wrote.

Harriet Tubman

Harriet Tubman also walked along these same streets. Tubman freed herself in 1849, but deeply missing her family, she determined to rescue them. Over the next year or so, she helped her niece, brother, and several others escape to Philadelphia.

But by 1850, with passage of the Fugitive Slave Act, it was too dangerous for the people she rescued to remain in the U.S. The next year she freed almost a dozen family and friends on Maryland’s Eastern Shore and then brought them all the way to Canada.

After a brutal winter there, she planned to return to Maryland, but needed money. So, “in the spring she returned to the States, and as usual earned money by working in hotels and families as a cook,” wrote Tubman’s friend, abolitionist Franklin F. Sanborn, in an 1863 newspaper article. “From Cape May, in the fall of 1852, she went back once more to Maryland, and brought away nine more fugitives.”

She may have also been working in Cape May at least two other summers, according to the New Jersey Historical Commission. From Cape May she could easily cross back to Maryland, very likely aided by the year round black community, which included many watermen.

Where she stayed and where she worked is not known. But Tubman had already made contact with the Philadelphia Underground Railroad, perhaps even knew Stephen Smith. She would have been comfortable at the intersection of Lafayette and Franklin.

Honoring These Civil Rights Icons

While today there is little at the intersection of Franklin and Lafayette Streets commemorating this history, that may soon change. The board of Macedonia Baptist Church recently signed an agreement with Bob and Zack Mullock, owners of the historic Chalfonte Hotel, to restore the church’s parsonage, once Quaker George Howell’s home.

They plan to create a museum honoring Harriet Tubman and highlighting Cape May’s ties to the Underground Railroad. It will also honor longtime church pastor, Rev. Robert Davis, who lectured about the contributions of African societies.

At the same time, the owners of the Stephen Smith house across the street are seeking funds to open it to the public to commemorate his role in the Underground Railroad.

And, although a recent fire seriously damaged the AME Church down the street, a local preservation group is working with its members to restore it. Perhaps one day it will again be a place not only for worship, but as a community meeting and performance space that will allow the area to resonate once again with discussions and debate on key issues of the day.