First Last Prev Next    No search results available
Bug 4620 - SEGV on updating credential for GRAM.
: SEGV on updating credential for GRAM.
Status: RESOLVED FIXED
: GRAM
gt2 Gram client
: 4.0.2
: PC Linux
: P3 normal
: 4.0.4
Assigned To:
:
:
:
:
  Show dependency tree / graph
 
Reported: 2006-07-26 05:12 by
Modified: 2007-02-14 14:53 (History)

Attachments
gt_refresh_credential.sh (14.48 KB, text/plain)
2006-07-26 05:14, tashiro 		Details
GRAM Protocol Patch (3.05 KB, patch)
2006-09-19 11:06, Joe Bester 		Details
gt_refresh_credential2.sh (16.36 KB, text/plain)
2006-09-21 00:40, tashiro 		Details
GRAM Protocol Patch (3.32 KB, patch)
2006-09-21 19:38, Joe Bester 		Details
Hide Obsolete (1) Add an attachment (proposed patch, testcase, etc.)


Note

You need to log in before you can comment on or make changes to this bug.

Description From 2006-07-26 05:12:55 
I met the SEGV on GT. conditions are listed below.

  - We created Multi-threaded application, which use GRAM.
  - We use refreshing credential mechanism,
    which execute below functions by individual thread periodically.
      globus_gss_assist_acquire_cred(),
      globus_gram_client_set_credentials()
  - I created the problem reproducing program.
    gt_refresh_credential.sh, which include C program and compile tool.
    I attach it.
  - OS is Linux (RH8), but the problem is occurred on Solaris too.
  - SEGV was occurred on Globus Toolkit 4.0.2.
    SEGV was not occurred on GT 2.4.3.
  - If I do not call globus_gram_client_set_credentials(),
    the SEGV was not occurred.

compile:
$ ./gt_refresh_credential.sh

execute:
$ globus-version
4.0.2
$ globus-hostname > hostlist
$ mv hostlist h ; cat h h h h > hostlist
$ ./gram_cred 1 8 `cat hostlist`
    0 sec: thread 00: start GRAM call to myhost x 8
    0 sec: thread 02: start GRAM call to myhost x 8
    0 sec: thread 01: start GRAM call to myhost x 8
    0 sec: thread 03: start GRAM call to myhost x 8
    0 sec: thread rf: start refresh credential (each 1 sec).
    1 sec: thread 00: 000 start.
    1 sec: thread rf: ref start.
    1 sec: thread 02: 000 start.
    1 sec: thread 01: 000 start.
    1 sec: thread 03: 000 start.
    1 sec: thread rf: ref       end.
Segmentation fault
$
debugger output:

Core was generated by `./gram_cred 1 8 myhost myhost myhost myhost'.
Program terminated with signal 11, Segmentation fault.


#0  0x401ce24d in SSL_new (ctx=0x80629ff) at ssl_lib.c:291
291             if (!s->method->ssl_new(s))
(gdb) where
#0  0x401ce24d in SSL_new (ctx=0x80629ff) at ssl_lib.c:291
#1  0x400fbcd9 in globus_i_gsi_gss_create_and_fill_context (
    minor_status=0x407a187c, context_handle_P=0x407a1884,
    cred_handle=0x80611f0, cred_usage=1, req_flags=4131)
    at globus_i_gsi_gss_utils.c:361
#2  0x400f7370 in gss_init_sec_context (minor_status=0x407a18e4,
    initiator_cred_handle=0x80611f0, context_handle_P=0x80785a8,
    target_name=0x8077438, mech_type=0x0, req_flags=4131, time_req=0,
    input_chan_bindings=0x0, input_token=0x0,
actual_mech_type=0x80785b4,
    output_token=0x407a18dc, ret_flags=0x807859c, time_rec=0x80785a0)
    at init_sec_context.c:128
#3  0x400938d3 in globus_l_xio_gsi_open_cb (op=0x8077fc8, result=0,
    user_arg=0x8078598) at globus_xio_gsi.c:1488
#4  0x400722f4 in globus_l_xio_driver_open_op_kickout
(user_arg=0x8077fc8)
    at globus_xio_driver.c:893
#5  0x4007e7af in globus_xio_driver_finished_open (in_dh=0x8077418,
    in_op=0x8077fc8, in_res=0) at globus_xio_pass.c:231
#6  0x400b331b in globus_l_xio_tcp_system_connect_cb (result=0,
    user_arg=0x80786a0) at globus_xio_tcp_driver.c:1797
#7  0x40084fa0 in globus_l_xio_system_kickout (user_arg=0x8078740)
    at globus_xio_system_select.c:1010
#8  0x401838c9 in globus_l_callback_thread_poll (user_arg=0x401a8a60)
    at globus_callback_threads.c:2482
#9  0x4019bf6c in globus_l_thread_pool_thread_start (user_arg=0x804f988)
    at globus_thread_pool.c:217
#10 0x4019af07 in thread_starter (temparg=0x804bc40)
    at globus_thread_pthreads.c:508
#11 0x40356881 in pthread_detach () from /lib/i686/libpthread.so.0
#12 0x420e40c7 in clone () from /lib/i686/libc.so.6
(gdb)
------- Comment #1 From 2006-07-26 05:14:22 ------- 
Created an attachment (id=1015) [details]
gt_refresh_credential.sh

The problem reproducing program.
------- Comment #2 From 2006-09-19 11:06:26 ------- 
Created an attachment (id=1048) [details]
GRAM Protocol Patch

I think this patch corrects this bug. Could you please try it out?
------- Comment #3 From 2006-09-21 00:37:58 ------- 
Thanks.
I patched my GT 4.0.3 and the program gram_cred was
successfully finished for 4 threads x 8times.
This patch looks fine.

But, unfortunately, My program was still failed.
This program execute globus_gram_client_job_refresh_credentials()
for all available job contacts.

I attach the program using this API. (gt_refresh_credential2.sh)
I executed the following command 10 times.

% globus-hostname > hostlist
% mv hostlist h; cat h h h h > hostlist
% ./gram_cred2 1 8 `cat hostlist`

I got 5 core files.
I always setup MALLOC_CHECK_=2 environment variable on
Linux environment, which affects glibc free().

If I remove this environment variable, the all gram_cred2 finished
successfully and no corefile was output.
(My program was successfully finished too.)

If I use GT 2.4.3 with MALLOC_CHECK_=2,
all gram_cred2 finished successfully too.

All corefiles on GT4.0.3 + patch with MALLOC_CHECK_=2 reports
following stack trace.

Core was generated by `./gram_cred2 1 8
myhost myhost myhost myhost'.
Program terminated with signal 6, Aborted.
#0  0x42028811 in kill () from /lib/i686/libc.so.6
(gdb) where
#0  0x42028811 in kill () from /lib/i686/libc.so.6
#1  0x40358fcd in raise () from /lib/i686/libpthread.so.0
#2  0x420284b4 in raise () from /lib/i686/libc.so.6
#3  0x42029bab in abort () from /lib/i686/libc.so.6
#4  0x42076d89 in posix_memalign () from /lib/i686/libc.so.6
#5  0x42075b85 in free () from /lib/i686/libc.so.6
#6  0x40091701 in globus_l_xio_gsi_handle_destroy (handle=0x81ba400)
    at globus_xio_gsi.c:695
#7  0x4009478a in globus_l_xio_gsi_close (driver_specific_handle=0x81ba400,
    attr=0x0, op=0x8147240) at globus_xio_gsi.c:1801
#8  0x40071b20 in globus_i_xio_driver_start_close (op=0x8147240, can_fail=0)
    at globus_xio_driver.c:702
#9  0x4008272d in globus_xio_driver_read_delivered (op=0x8097e98, in_ndx=0,
    deliver_type=0x407a181c) at globus_xio_pass.c:1506
#10 0x400716a1 in globus_l_xio_driver_op_read_kickout (user_arg=0x8097e98)
    at globus_xio_driver.c:626
#11 0x40081965 in globus_xio_driver_finished_read (in_op=0x8097e98,
    result=662, nbytes=0) at globus_xio_pass.c:1227
#12 0x40095297 in globus_l_xio_gsi_read_cb (op=0x8097e98, result=662,
    nbytes=0, user_arg=0x81ba400) at globus_xio_gsi.c:2063
#13 0x40071688 in globus_l_xio_driver_op_read_kickout (user_arg=0x8097e98)
    at globus_xio_driver.c:620
#14 0x40081965 in globus_xio_driver_finished_read (in_op=0x8097e98,
    result=661, nbytes=0) at globus_xio_pass.c:1227
#15 0x400b45da in globus_l_xio_tcp_finish_read (handle=0x8157d48, result=661,
    nbytes=0) at globus_xio_tcp_driver.c:2196
#16 0x400b4663 in globus_l_xio_tcp_system_read_cb (result=661, nbytes=0,
    user_arg=0x8157d48) at globus_xio_tcp_driver.c:2211
#17 0x40084fea in globus_l_xio_system_kickout (user_arg=0x817d140)
    at globus_xio_system_select.c:1016
#18 0x401838c9 in globus_l_callback_thread_poll (user_arg=0x401a8a60)
    at globus_callback_threads.c:2482
#19 0x4019bf6c in globus_l_thread_pool_thread_start (user_arg=0x804ffc0)
    at globus_thread_pool.c:217
#20 0x4019af07 in thread_starter (temparg=0x804c300)
    at globus_thread_pthreads.c:508
#21 0x40356881 in pthread_detach () from /lib/i686/libpthread.so.0
#22 0x420e40c7 in clone () from /lib/i686/libc.so.6
(gdb)

Thanks.
------- Comment #4 From 2006-09-21 00:40:38 ------- 
Created an attachment (id=1053) [details]
gt_refresh_credential2.sh

This is the new error reproducing program, 
which use globus_gram_client_job_refresh_credentials().
------- Comment #5 From 2006-09-21 19:38:49 ------- 
Created an attachment (id=1056) [details]
GRAM Protocol Patch

Attaching a new version of this patch. This catches the case where odd things
can happen when EOF occurs during delegation. There's a similar patch for bug
#4726 which might also help.

joe
------- Comment #6 From 2006-09-22 03:27:54 ------- 
Thanks for your quick fix.

This patch (with Bug 4726) fixed the problem.
The second program gram_cred2 finished successfully by
4 threads, 64times for each run, times 10 runs.

My original program was no-error too.

This patch is sufficient for me.

Thanks.
------- Comment #7 From 2006-09-22 08:48:41 ------- 
I've committed (a slightly modified version of) the patch to the 4.0 branch and
CVS trunk. The advisory/update package is available at
http://www.globus.org/toolkit/advisories.html?version=4.0#globus_gram_protocol-6.5

joe
------- Comment #8 From 2006-10-02 18:58:40 ------- 
*** Bug 4689 has been marked as a duplicate of this bug. ***
First Last Prev Next    No search results available