Bug 2967 - Job submission inconsistent behaviour with delegated credential
: Job submission inconsistent behaviour with delegated credential
Status: RESOLVED WORKSFORME
: GRAM
wsrf managed execution job service
: 3.9.5
: PC Linux
: P1 normal
: 4.0.1
Assigned To:
:
:
:
:
  Show dependency treegraph
 
Reported: 2005-03-17 03:53 by
Modified: 2006-03-30 09:54 (History)


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2005-03-17 03:53:18
I created a 5 second delegate credential, as follows:

./bin/globus-credential-delegate -l 5 5sec

Then the first time I tried to use this to submit,
./bin/globusrun-ws -submit -f test_super_simple.xml -job-credential-file 5sec
I get the following exception:

Submitting job...Failed.
globusrun-ws: Error submitting job
globus_soap_message_module: SOAP Fault
Fault code: soapenv:Server.userException
Fault string: java.rmi.RemoteException: Job creation failed.; nested exception 
is:
        java.lang.RuntimeException: Couldn't obtain a delegated credential.


But then on when I submit job, it completes successfully.
On investigation found that, the job is using the user's credential.
It should have failed on all job submissions using that delegated credential.

The proxy was destroyed , and the job was submitted again, the following error 
was got:
./bin/globusrun-ws -submit -f test_super_simple.xml -job-credential-file 5sec
Submitting job...Failed.
globusrun-ws: Error submitting job
globus_gsi_gssapi: Error with gss credential handle
globus_credential: Valid credentials could not be found in any of the possible 
locations specified by the credential search order.
Valid credentials could not be found in any of the possible locations 
specified by the credential search order.

Attempt 1

globus_credential: Error reading host credential
globus_sysconfig: Could not find a valid certificate file: The host cert could 
not be found in:
1) env. var. X509_USER_CERT
2) /etc/grid-security/hostcert.pem
3) $GLOBUS_LOCATION/etc/hostcert.pem
4) $HOME/.globus/hostcert.pem

The host key could not be found in:
1) env. var. X509_USER_KEY
2) /etc/grid-security/hostkey.pem
3) $GLOBUS_LOCATION/etc/hostkey.pem
4) $HOME/.globus/hostkey.pem



Attempt 2

globus_credential: Error reading proxy credential
globus_sysconfig: Could not find a valid proxy certificate file location
globus_sysconfig: Error with key filename
globus_sysconfig: File does not exist: /tmp/x509up_u50806 is not a valid file

Attempt 3

globus_credential: Error reading user credential
globus_credential: Key is password protected: GSI does not currently support 
password protected private keys.
OpenSSL Error: pem_lib.c:401: in library: PEM routines, function 
PEM_do_header: bad password read
------- Comment #1 From 2005-03-23 15:36:22 -------
The job credential file will be a file that is created by the gram service for
use by your application.  The 
gram service does not need a job-credential-file to submit and start your user
application.  gram will 
submit your jobs fine, but if you deleated a short lived cred for your job,
then your job will likely fail.

I'm confused by attempt 1, 2 and 3.  What commands are you running there?  What
have you done to 
your environment before running those commands?
------- Comment #2 From 2005-03-30 00:35:31 -------
These are the steps I followed:

1.I create a delegated credential with validity of 5seconds, by using the 
command:
./bin/globus-credential-delegate -l 5 5sec

2.I submit the jib using the above created credential:

./bin/globusrun-ws -submit -f test.xml -job-credential-file 
5sec                                Submitting job...Failed.
globusrun-ws: Error submitting job
globus_soap_message_module: SOAP Fault
Fault code: soapenv:Server.userException
Fault string: java.rmi.RemoteException: Job creation failed.; nested exception 
is:
        java.lang.RuntimeException: Couldn't obtain a delegated credential.

3. I can submit the job using the credential created in step 1,
./bin/globusrun-ws -submit -f test.xml -job-credential-file 5sec
Submitting job...Done.
Job ID: uuid:884d4844-a0e5-11d9-a7c2-00025589a5dd
Termination time: 03/31/2005 06:32 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.


But this time it succeeds, but it should have failed since the credential 
created  in step 1 has expired.


4. I destroy the user proxy
grid-proxy-destroy

5.I again submit the job,
 ./bin/globusrun-ws -submit -f test.xml -job-credential-file 5sec
These are the exceptions got:


Submitting job...Failed.
globusrun-ws: Error submitting job
globus_gsi_gssapi: Error with gss credential handle
globus_credential: Valid credentials could not be found in any of the possible 
locations specified by the credential search order.
Valid credentials could not be found in any of the possible locations 
specified by the credential search order.

Attempt 1

globus_credential: Error reading host credential
globus_sysconfig: Could not find a valid certificate file: The host cert could 
not be found in:
1) env. var. X509_USER_CERT
2) /etc/grid-security/hostcert.pem
3) $GLOBUS_LOCATION/etc/hostcert.pem
4) $HOME/.globus/hostcert.pem

The host key could not be found in:
1) env. var. X509_USER_KEY
2) /etc/grid-security/hostkey.pem
3) $GLOBUS_LOCATION/etc/hostkey.pem
4) $HOME/.globus/hostkey.pem



Attempt 2

globus_credential: Error reading proxy credential
globus_sysconfig: Could not find a valid proxy certificate file location
globus_sysconfig: Error with key filename
globus_sysconfig: File does not exist: /tmp/x509up_u500 is not a valid file

Attempt 3

globus_credential: Error reading user credential
globus_credential: Key is password protected: GSI does not currently support 
password protected private keys.
OpenSSL Error: pem_lib.c:401: in library: PEM routines, function 
PEM_do_header: bad password read






------- Comment #3 From 2005-03-30 00:42:06 -------
6.I recreate the proxy 
grid-proxy-init

7. I resubmit the job , and it succeeds
Submitting job...Done.
Job ID: uuid:5c0fb694-a0e6-11d9-b5c7-00025589a5dd
Termination time: 03/31/2005 06:38 GMT
Current job state: CleanUp
Current job state: Done
Destroying job...Done.

I have only set the following environment variables:
GRIDMAP
X509_CERT_DIR
GRID_SECURITY_DIR






------- Comment #4 From 2005-11-02 19:19:59 -------
There are relevant and irrelevant pieces to this.  The fact that subsequent
submissions after an initial job failed to get the delegated credential is
clearly not right.  It looks to me like a service-side bug so I'm
re-categorizing this bug.

The fact that for some reason you destroy your user proxy is completely
unrelated.  If you have no user proxy, then you can't even contact the GRAM
factory to submit a job.  If there's an inconsistency in the error messages
there, then a separate bug should be submitted for the client.
------- Comment #5 From 2006-03-30 09:54:06 -------
I can't seem to reproduce this with the latest 4.0.x development code. 4.0.2
should be coming out fairly soon. If you're still having problems, try
upgrading at that point to see if this problem goes away. If it does, feel free
to reopen the bug. I am considering this bug only about the delegation issue,
not the proxy issue. Again, you should submit a separate bug for the proxy
issue if you haven't already.