Dolley was the heartbeat of the White House during the Madison administration. Visitors remarked on her vivacity, social grace, and charm. The more reserved James happily let Dolley take the lead at important social events, which she frequently hosted. Her elegant weekly “drawing rooms” found a tactful place between the cold formality of Martha Washington’s “levees” and the widowed Thomas Jefferson’s heedless informality. As political as they were social, the events facilitated political and diplomatic connections and reduced temperatures in the overwrought city.

When James and Dolley married, her son from her first marriage, John Payne “Payne” Todd, was still a toddler. Still in school when Madison became president, Payne spent little time in the White House. Although the Madisons had no children or grandchildren living permanently with them, the White House hosted many from their extended family. Dolley’s sister, Anna Payne Cutts and her husband Richard, with their three children, initially lived with the Madisons in 1809. Even after they moved out, the family made a frequent presence at the White House, given the closeness of Dolley and Anna. Madison’s nephew, Robert, the son of his brother William, lived in the White House for about six months part way through Madison’s presidency. Edward Coles, the president’s private secretary and Dolley’s cousin, occupied a room.

The White House bustled with free and enslaved staff. John Sioussat worked as the White House steward. Like the two preceding Virginia presidents, the Madisons kept probably at least a dozen enslaved workers at the executive mansion at any given time, including footman/valet Paul Jennings, maid Susan or “Sukey”, and coachman Joseph Bolden. Bolden’s wife, Milley, owned by Dolley’s cousin Francis Scott Key, and their child joined the presidential household after the Boldens purchased freedom for their entire family. John Freeman and his free wife Melinda also worked there.

Madison hired other enslaved workers from around the city from time to time. Dolley also borrowed the labor of slaves from friends and associates when she needed additional workers for special occasions. Jennings was among other unnamed staff who famously helped save George Washington’s life-size portrait from flames when the British burned the White House. Surrounded by enslaved staff in the nation’s nerve center of freedom, Edward Coles often lectured Madison about the contradictions of republicanism and slavery. We can only imagine Madison’s sympathetic but ultimately dismissive responses.