Bugzilla – Bug 4178
no job output when streaming with globusrun-ws
Last modified: 2006-10-18 16:46:57
You need to log in before you can comment on or make changes to this bug.
On Teragrid, I submit a streaming job and there is no output displayed on the terminal. If I submit a job that echo info; sleep 30; echo info I can see the data in the stdout file in my home dir, but the stdout data is never displayed to the terminal. Looking at the gridftp log file, the only commands that show up for the job are delete commands for the job's stdout and stderr. So either the globusrun-ws commands are failing (silently) or not being attempted for some reason. Host: tg-grid1.uc.teragrid.org GLOBUS_LOCATION: /soft/globus-4.0.1-r2/ container log file: /usr/local/logs/globus_container.tg-grid1.8443.log Sample command: tg-grid1 /home/smartin> globusrun-ws -submit -s -c /bin/hostname globusrun-ws -dbg output: /home/smartin/stream_dbg_output
Mike Showerman reports that it works fine on other sites. Grid1 is sles8/ia64
JP and I have reproduced this twice sofar. After restarting the GT4 container, the first job attempt works fine and the stdout data is displayed. Subsequent jobs do not show the stdout data. smartin@tg-grid1:~> globusrun-ws -submit -s -c /bin/hostname Delegating user credentials...Done. Submitting job...Done. Job ID: uuid:1a26ed0e-8f58-11da-98cd-0007e9d81215 Termination time: 01/28/2006 17:12 GMT Current job state: Active Current job state: CleanUp-Hold tg-grid1 Current job state: CleanUp Current job state: Done Destroying job...Done. Cleaning up any delegated credentials...Done. smartin@tg-grid1:~> globusrun-ws -submit -s -c /bin/hostname Delegating user credentials...Done. Submitting job...Done. Job ID: uuid:3da438fe-8f58-11da-b857-0007e9d81215 Termination time: 01/28/2006 17:13 GMT Current job state: Active Current job state: CleanUp-Hold Current job state: CleanUp Current job state: Done Destroying job...Done. Cleaning up any delegated credentials...Done.
I added some debug statements to globusrun-ws and it looks like sometimes the stdout/stderr resource properties are not populated when globusrun-ws does GetMultipleResourceProperties. Since neither stdout or stderr are returned, streaming doesn't happen. Any idea why this is happening on the java side? joe
If globusrun-ws tries to grab those RPs immediately after the job is submitted then I could see this happening. I would suggest waiting until after a notification occurs before trying to grab those values if this is the way globusrun-ws behaves.
It checks the property after the ACTIVE response is returned from the container. joe
Created an attachment (id=835) [details] Patch setting stdout/err RPs before going Active Although I can't reproduce this myself, there is clearly a race condition here since the Active state is set before the OpenStdout and OpenStderr internal states are processed. I'm attaching a patch that simply moves the setting of the state to the end of the processOpenStderrState() method in the state machine.
The fix has been comitted to the globus_4_0_branch. If somebody who can reliably reproduce this problem could apply the patch and report back that would be great.
Since there's been no response on this in terms of verifying the patch, I'm resolving this as FIXED. Feel free to reopen if there's still a problem with version 4.0.2 or above.