I contacted Joost and he showed me where the port address can be provided. The params of the connect is where you assign the port. If you look at pqconnection.pp you will see:
procedure TPQConnection.DoInternalConnect;
var msg : string;
begin
{$IfDef LinkDynamically}
InitialisePostgres3;
{$EndIf}
inherited dointernalconnect;
FConnectString := '';
if (UserName <> '') then FConnectString := FConnectString + ' user=''' + UserName + '''';
if (Password <> '') then FConnectString := FConnectString + ' password=''' + Password + '''';
if (HostName <> '') then FConnectString := FConnectString + ' host=''' + HostName + '''';
if (DatabaseName <> '') then FConnectString := FConnectString + ' dbname=''' + DatabaseName + '''';
if (Params.Text <> '') then FConnectString := FConnectString + ' '+Params.Text;
FSQLDatabaseHandle := PQconnectdb(pchar(FConnectString));
The Params.Text would need to contain something like "port=????"
Hope this helps.