He gained very varied experience in the next 20 years and in December 1737, as relations with Spain worsened, was appointed Commodore commanding at Jamaica. When the War of Jenkins's Ear broke out he was superseded by Admiral Edward ('Grog') Vernon and was his second-in-command at the celebrated capture of Portobello 'with only six ships', as shown here. Brown's 'Hampton Court' led the bombardment of the (rather ill-manned) Iron Castle, and when the Spanish commander offered surrender and his sword to Brown, the latter declined on the grounds that accepting them was Vernon's prerogative. When Brown took the Spaniard to him, Vernon accepted the surrender and responded to the the request that Brown have the sword, in compliment to the vigour of his assault, by first receiving it but then immediately giving it to him. In 1740, while Vernon continued his campaign on the Spanish Main – notably, though unsuccessfully, at Cartagena – Brown commanded at Jamaica in his stead. There his health quickly deteriorated and he arrived back in England in October 1740.

In 1741 he held two brief home commands before being appointed resident Navy Board Commissioner of Chatham Dockyard, on 27th March 1742, and remained an effective holder of the post to his death on 23rd March 1753. He was buried at St Mary's, Chatham, on the 27th March, the eleventh anniversary of his appointment. One of Brown's daughters (Lucy) married Admiral William Parry and her daughter married Captain William Locker, Nelson's early commander and friend, and later Lieutenant-Governor of Greenwich Hospital (d.1800). William's son Edward Hawke Locker (1777–1849) was from 1819 Secretary and later senior Commissioner of the Hospital and in 1824 founded the Naval Gallery in its Painted Hall (an earlier but unachieved idea of his father, a patron of artists). In 1838 he presented this portrait to it and also commissioned and presented George Chambers' interpretation of the attack on Portobello (BHC0355), to commemorate its centenary in 1839.

The portrait relates to a mezzotint by John Faber dated 1740 (PAD4583), which shows the sitter as older and more ruggedly characterful in facial appearance, with the background scene both closer and enlarged, and the sword as a much more realistic short hanger rather than with a long cutlass blade. While previously assumed to be the source of the print the painting may therefore, on general appearances, be derived from it as a posthumous and idealised image, perhaps made for William Locker. The Spanish sword Brown received at Portobello has also been in the Museum collection since 1963 (WPN1248) when it was purchased from a later Locker descendant.