Connecting to the VSS WebService on connections other than port 80
When connecting to VSS over the internet to a server that has the VSS web service on a web site that isn't sitting on port 80, the full name of the service needs to be specified in the connection settings on the client side, as shown below. If you simply try to specify http(s)://x.x.x.x:y (where y is the port number), Visual Studio will somehow miss the port when it parses out the server's IP address. To fix the problem, use the full path to the webservice, such as http://x.x.x.x:y/SourceSafe/VssService.asmx. Visual Studio will pick up the port number correctly when the URL is specified this way.
To set up internet access to VSS on a web site that isn't the default web site on the server side, I used the the VSS Admin exe to setup internet access according
to this article, backup the ISS virtual directory that SourceSafe created, and then restored it to the website on the arbitrary port.