Vernon began his career as Private Secretary to King Charles II's eldest illegitimate son, James Scott, Duke of Monmouth. In 1677 James Vernon met the artist Godfrey Kneller, who had just arrived in England from Germany. Vernon became Kneller's patron, paying the painter's rent in return for this portrait. He introduced Kneller to the Duke of Monmouth, and Kneller noted that Vernon could 'introduce [me] to courtiers, for I take him for an honest and most charming gentleman, as all who know him say'. Consequently, this portrait helped launch a long and successful career for Kneller, who became the principal painter in Britain, serving the King and being awarded a knighthood. Although Vernon and Monmouth were close, Vernon left the Duke's service when the latter went into exile in 1684 and subsequently transferred his loyalties to William and Mary of Orange.