Bugzilla – Bug 4861
Problem with HTTP chunked mode transfer in globus-url-copy
Last modified: 2007-03-20 15:24:27
You need to log in before you can comment on or make changes to this bug.
This is identical to bug #703 which is supposed to be resolved, but HTTP chunked transfers still fails with globus-url-copy in GT 4.0.3. PROBLEM DESCRIPTION: When I use globus-url-copy to transfer a file to a HTTP location, the transfer fails because of an incorrect implementation of chunked HTTP transfer (in what I think is globus_gass_transfer_http.c). E.g. globus-url-copy file:///u/drlead/.bashrc http://hostname:53000/mybashrc When looking at the bytes sent on the wire, the last chunk that globus-url-copy sends is: 0[CRLF] 0[CRLF] while the HTTP/1.1 specification requires the transfer to end with a 0 byte chunk, followed by an extra trailing [CRLF]. i.e., the trailing [CRLF] is missing when globus-url-copy does the transfer. Instead, the above should be: 0[CRLF] 0[CRLF] [CRLF] or: 0[CRLF] [CRLF] Also, many HTTP servers do not like a zero sized chunk which is not the last chunk. So while: 0[CRLF] 0[CRLF] 0[CRLF] [CRLF] is a valid way to end the chunk according to the HTTP/1.1 specification, most HTTP servers expect a 0 byte chunk to be the last chunk and expect the trailing [CRLF} right afterwards. So ending with just: 0[CRLF] [CRLF] may make it more interoperable. REFERENCE: http://www.ietf.org/rfc/rfc2616.txt section 3.6.1: Chunked Transfer Coding http://bugzilla.globus.org/globus/show_bug.cgi?id=703 VERSIONS: *) GT4 built on Redhat Linux x86_64 from gt4.0.3-all-source-installer. (Same error seen with GT 4.0.1 also) *) [gt4.0.3-all-source-installer/] head source-trees/gass/transfer/source/library/globus_gass_transfer_http.c <snip/> /** * @file globus_gass_transfer_http.c http/https Protocol Module Implementation * * This module implements the http and https URL schemes for the GASS transfer * library * * CVS Information: * $Source: /home/globdev/CVS/globus-packages/gass/transfer/source/library/globus_gass_transfer_http.c,v $ * $Date: 2005/04/18 23:01:21 $ * $Revision: 1.41 $ * $Author: mlink $ */ [/usr/local/globus/bin]$ ./globus-url-copy -versions globus-url-copy: 3.20 (1133498504-63) globus_ftp_client_restart_plugin: 3.4 (1138746361-63) globus_ftp_client_debug_plugin: 3.4 (1138746361-63) globus_ftp_client_perf_plugin: 3.4 (1138746361-63) globus_ftp_client_throughput_plugin: 3.4 (1138746361-63) globus_ftp_control: 2.3 (1113865369-1) globus_ftp_client: 3.4 (1138746361-63) globus_xio_gsi: 0.34 (1138825633-63) globus_xio_tcp: 0.34 (1138825633-63) globus_xio_system_select: 0.34 (1138825633-63) globus_xio_file: 0.34 (1138825633-63) globus_xio: 0.34 (1138825633-63) globus_io: 6.0 (1113865852-1) globus_gsi_callback_module: 0.23 (1121890023-63) globus_credential: 1.9 (1155324425-63) globus_gsi_proxy: 2.4 (1113610195-1) globus_gsi_openssl_error: 0.12 (1154020771-63) globus_openssl: 0.6 (1113610167-1) globus_gsi_gssapi: 4.9 (1130374406-63) globus_sysconfig: 1.20 (1155324405-63) globus_callout_module: 0.6 (1113865156-1) globus_gss_assist: 3.22 (1140210636-63) globus_extension_module: 7.20 (1155918516-63) globus_callback_nonthreaded: 7.20 (1155918516-63) globus_object: 7.20 (1155918516-63) globus_error: 7.20 (1155918516-63) globus_common: 7.20 (1155918516-63) globus_gass_transfer: 2.10 (1136908920-63) globus_gass_copy: 3.20 (1133498504-63) globus_thread_common: 7.20 (1155918516-63) globus_thread_none: 7.20 (1155918516-63) HTTP Servers tried for target location: Apache Tomcat 5.5 and Simple Framework HTTP Server 3.1.3 STACK TRACE: *) Error seen on Catalins.out log: java.io.IOException: Invalid CRLF at org.apache.coyote.http11.filters.ChunkedInputFilter.parseCRLF(ChunkedInputFilter.java:301) at org.apache.coyote.http11.filters.ChunkedInputFilter.parseEndChunk(ChunkedInputFilter.java:320) at org.apache.coyote.http11.filters.ChunkedInputFilter.doRead(ChunkedInputFilter.java:133) at org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:712) at org.apache.coyote.Request.doRead(Request.java:418) at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:284)
Created an attachment (id=1184) [details] Patch to globus_gass_transfer Attaching a patch containing a proposed fix to this issue.
This fix is committed to CVS. joe