between Hilsea and Portsmouth” might imply the Green Post or the Fountain, or a pub even further south at Portsea.

Quote

A general Court Martial assembled at Portsmouth, on Friday, for the trial of several soldiers of the Inverness militia, for rioting at a public-house between Hilsea and Portsmouth, and striking an officer of the 42d regiment, in the execution of his duty; Lieutenant-Colonel Carne, East Essex, President.

— Caledonian Mercury, Saturday 09 January 1813 source

The name of the pub may be in records of the court martial, if they still exist. I’m sure I’ve read a more detailed account of a court martial related to a Hilsea pub - something about treasonous speech - but I can’t put my hand on it right now.

Some possible leads: