Ponsonby was the orphaned daughter of an upper class family of the Irish Protestant Ascendancy. Her mother died when she was three; her father remarried, but himself died when she was seven. Her stepmother then remarried but she herself died when Ponsonby was 13. Her stepfather took no interest in the now destitute child, and she was fostered by her father's cousin, Lady Betty Fownes, although she spent most of her time at boarding school in Kilkenny.
The Fowneses were neighbors of the aristocratic Butler Family, and in spite of the age difference, Sarah fell in love with Lady Eleanor Butler (1745?-1829). The relationship was not approved of by their families, and in 1778 the two women eloped to Wales, eventually settling in Llangollen, at Pen y Maes Cottage, in 1780. Their relationship caused local curiosity but any initial hostility and gossip (they dressed in men's clothing) were soon overcome, and Butler's independent income enabled them to be local benefactors of the poor and to offer hospitality to a long succession of famous literati and others for many years, including Lord Byron, the Duke of Wellington, Sir Walter Scott, Thomas De Quincy, William Wordsworth and Robert Southey. They remained at the house, which they renamed Plas Newydd and considerably enlarged, until their deaths.
The "Ladies of Llangollen" remain two of the most famous personalities in Welsh history.