An underground conduit with a vaulted stone ceiling carries fresh water from a dammed creek in Soldiers' Gully to the Baths, and then under Quality Row to spill into the stream that runs through the Common into Emily Bay. It is said that the stonemasons required to build the Baths and conduits were deliberately convicted and transported from England to Norfolk Island.