Bug 4344

Does this happen when the 3pt is between two seperate servers?

humm interesting it actually works when the servers are different.

That's a known side-affect of the connection caching, which is keyed on the
server contact string.  It won't be changed.  If you really need to transfer
files between the same server for testing purposes or whatever, you can avoid
the issue by using the hostname for one and the ip address for the other, or
something like that; so long as the actual host:port strings are different.

Mike

(In reply to comment #3)
> That's a known side-affect of the connection caching, which is keyed on the
> server contact string.  

Is this documented somewhere.

It won't be changed.  If you really need to transfer
> files between the same server for testing purposes or whatever, you can avoid
> the issue by using the hostname for one and the ip address for the other, or
> something like that; so long as the actual host:port strings are different.
> 

I dont agree that a user should be handling this or worry about why his
transfers are failing just cause he chose to do a tpt on the same gridftp
server.

1) The error thrown does not tell a user why this happens. I spent nearly half
a day figuring out why this was happening. 

2) If you can't map 2 connections in your map then you could in the client
automatically use the ip for one connection and the hostname for the other
rather then expecting the user to do the same

3) IMO the client should continue working albeit inefficently. If you cant do
solution 2 then atleast setting notpt automatically on these transfers will
also solve the problem.

_Gaurang


> Mike
> 

(In reply to comment #3)
> That's a known side-affect of the connection caching, which is keyed on the
> server contact string.  It won't be changed.  If you really need to transfer
> files between the same server for testing purposes or whatever, you can avoid
> the issue by using the hostname for one and the ip address for the other, or
> something like that; so long as the actual host:port strings are different.
> 
> Mike
> 

The failures are due to the server configuration, not the client making
multiple connections.  It would fail the same way if you were only making one
connection and happened to have two seperate people trying to connect to the
server.

I don't think it was documented, but now it is documented here.

let me clarify the situation.

Assume, the limit for server connections is 2.

I can understand if multiple simultaneous g-u-c instances failed with such a
server configuration.

but this is not the problem i am describing

A single g-u-c should be able to perform multiple transfers from this server to
itself in 3rd party mode. 

But this fails.... whereas it should have succeeded, since this particular
client holds both connections to the server.

A general application may have no idea that the src and destination urls are on
the same host. 

Note : the first transfer succeeds and the subsequent ones are failing. This is
not a consistent behaviour. Either all fail (which IMO should not because it
can be trivially handled in the client)  or all should succeed (which is what i
am rooting for).

I dont want to argue for arguing sake just that a user should not have to
handle this.

I looked at this further and do see a bug that wasn't expected.  (I apoligize
if you said this in your earlier comments and I missed it) The client will
create a new connection for each file as I expected, but it never closes the
old connections, so it uses far more than the 3 connections that I assumed (the
extra one that covered the overlap between opening and closing either the
source or dest connection).  I will fix that, and it should give you the result
that a 3pt to the same server only takes 2 connections.

This won't make it into 4.0.2 though, as that had already been close to release
earlier in the week.

Mike  

Thanks Mike.

Ok so from what you say i understand that while you cannot/or wont handle the
reuse of the connections (if the src-dest are same) atleast it wont take more
the 2 connections at any given time. Which is fine if not optimized.

I am a little worried that it wont make it to 4.0.2 since this would have
delivered the fix to a wider audience. I have always found putting advisories
on binary installs a pain.

Is there a advisory or patch for this i can build against 4.0.2
It would be good to get this patch and the other one you fixed in the same
advisory bundle.

-Gaurang

There is no fix for this yet, but it would be a seperate advisory.  This is due
to a bug in the ftp client library package, not the globus-url-copy package.

Created an attachment (id=1182) [details]
patch to gridftp/client/source/globus_ftp_client_handle.c

This fixes the connection leak when the source and dest have the same host:port
strings.

This fix was included in the 4.0.4 release.