Bug 4861 - Problem with HTTP chunked mode transfer in globus-url-copy
: Problem with HTTP chunked mode transfer in globus-url-copy
Status: RESOLVED FIXED
: GridFTP
globus-url-copy
: 4.0.1
: All All
: P3 normal
: 4.0.5
Assigned To:
:
:
:
:
  Show dependency treegraph
 
Reported: 2006-11-16 17:40 by
Modified: 2007-03-20 15:24 (History)


Attachments
Patch to globus_gass_transfer (7.75 KB, patch)
2007-02-07 13:37, Joe Bester
Details


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2006-11-16 17:40:25
This is identical to bug #703 which is supposed to be resolved, but HTTP
chunked transfers still fails with globus-url-copy in GT 4.0.3.

PROBLEM DESCRIPTION:
When I use globus-url-copy to transfer a file to a HTTP location, the transfer
fails because of an incorrect implementation of chunked HTTP transfer (in what
I think is globus_gass_transfer_http.c).
E.g. globus-url-copy file:///u/drlead/.bashrc http://hostname:53000/mybashrc

When looking at the bytes sent on the wire, the last chunk that globus-url-copy
sends is:
0[CRLF]
0[CRLF]

while the HTTP/1.1 specification requires the transfer to end with a 0 byte
chunk, followed by an extra trailing [CRLF]. i.e., the trailing [CRLF] is
missing when globus-url-copy does the transfer. Instead, the above should be:
0[CRLF]
0[CRLF]
[CRLF]

or:
0[CRLF]
[CRLF]

Also, many HTTP servers do not like a zero sized chunk which is not the last
chunk. So while:
0[CRLF]
0[CRLF]
0[CRLF]
[CRLF]

is a valid way to end the chunk according to the HTTP/1.1 specification, most
HTTP servers expect a 0 byte chunk to be the last chunk and expect the trailing
[CRLF} right afterwards. So ending with just:
0[CRLF]
[CRLF]

may make it more interoperable.

REFERENCE:
http://www.ietf.org/rfc/rfc2616.txt section 3.6.1: Chunked Transfer Coding
http://bugzilla.globus.org/globus/show_bug.cgi?id=703

VERSIONS:
*) GT4 built on Redhat Linux x86_64 from gt4.0.3-all-source-installer. (Same
error seen with GT 4.0.1 also)
*) [gt4.0.3-all-source-installer/] 
head source-trees/gass/transfer/source/library/globus_gass_transfer_http.c

<snip/>
/**
 * @file globus_gass_transfer_http.c http/https Protocol Module Implementation
 *
 * This module implements the http and https URL schemes for the GASS transfer
 * library
 *
 * CVS Information:
 * $Source:
/home/globdev/CVS/globus-packages/gass/transfer/source/library/globus_gass_transfer_http.c,v
$
 * $Date: 2005/04/18 23:01:21 $
 * $Revision: 1.41 $
 * $Author: mlink $
 */

[/usr/local/globus/bin]$ ./globus-url-copy -versions
globus-url-copy: 3.20 (1133498504-63)
globus_ftp_client_restart_plugin: 3.4 (1138746361-63)
globus_ftp_client_debug_plugin: 3.4 (1138746361-63)
globus_ftp_client_perf_plugin: 3.4 (1138746361-63)
globus_ftp_client_throughput_plugin: 3.4 (1138746361-63)
globus_ftp_control: 2.3 (1113865369-1)
globus_ftp_client: 3.4 (1138746361-63)
globus_xio_gsi: 0.34 (1138825633-63)
globus_xio_tcp: 0.34 (1138825633-63)
globus_xio_system_select: 0.34 (1138825633-63)
globus_xio_file: 0.34 (1138825633-63)
globus_xio: 0.34 (1138825633-63)
globus_io: 6.0 (1113865852-1)
globus_gsi_callback_module: 0.23 (1121890023-63)
globus_credential: 1.9 (1155324425-63)
globus_gsi_proxy: 2.4 (1113610195-1)
globus_gsi_openssl_error: 0.12 (1154020771-63)
globus_openssl: 0.6 (1113610167-1)
globus_gsi_gssapi: 4.9 (1130374406-63)
globus_sysconfig: 1.20 (1155324405-63)
globus_callout_module: 0.6 (1113865156-1)
globus_gss_assist: 3.22 (1140210636-63)
globus_extension_module: 7.20 (1155918516-63)
globus_callback_nonthreaded: 7.20 (1155918516-63)
globus_object: 7.20 (1155918516-63)
globus_error: 7.20 (1155918516-63)
globus_common: 7.20 (1155918516-63)
globus_gass_transfer: 2.10 (1136908920-63)
globus_gass_copy: 3.20 (1133498504-63)
globus_thread_common: 7.20 (1155918516-63)
globus_thread_none: 7.20 (1155918516-63)

HTTP Servers tried for target location: 
Apache Tomcat 5.5 and Simple Framework HTTP Server 3.1.3

STACK TRACE:
*) Error seen on Catalins.out log:
java.io.IOException: Invalid CRLF
        at
org.apache.coyote.http11.filters.ChunkedInputFilter.parseCRLF(ChunkedInputFilter.java:301)
        at
org.apache.coyote.http11.filters.ChunkedInputFilter.parseEndChunk(ChunkedInputFilter.java:320)
        at
org.apache.coyote.http11.filters.ChunkedInputFilter.doRead(ChunkedInputFilter.java:133)
        at
org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:712)
        at org.apache.coyote.Request.doRead(Request.java:418)
        at
org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:284)
------- Comment #1 From 2007-02-07 13:37:20 -------
Created an attachment (id=1184) [details]
Patch  to globus_gass_transfer

Attaching a patch containing a proposed fix to this issue.
------- Comment #2 From 2007-03-20 15:24:27 -------
This fix is committed to CVS.

joe