- Conscience; an ethical essay (1773)
- On Illicit Love. Written Among the Ruins of Godstow Nunnery, Near Oxford (1775)
- Observations on the popular antiquities of Great Britain: Including the Whole of Mr. Bourne's Antiquitates Vulgares (1777)
- The History and Antiquities of the Town and County of Newcastle-upon-Tyne (1789), in 2 vols.
- Political observations on the Test Act (1790)
- A defence of the pamphlet ascribed to John Reeves, Esq., and entitled, "Thoughts on the English government" (1796)
- * "Account of Inscriptions discovered on the Walls of an Apartment in the Tower of London" in Archaeologia, vol. 13 (1800), pp.68–09