Bug 2955 - Job submission fails when container is started from non GLOBUS_LOCATION
: Job submission fails when container is started from non GLOBUS_LOCATION
Status: RESOLVED FIXED
: Java WS Security
Authentication
: 3.9.5
: PC Windows 2000
: P3 normal
: ---
Assigned To:
:
:
:
:
  Show dependency treegraph
 
Reported: 2005-03-16 02:32 by
Modified: 2005-04-09 10:53 (History)


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2005-03-16 02:32:58
Job submission fails when the container is started from non GLOBUS_LOCATION.
This issue is directly related to bugzilla 2899 "relative path does not work for
credentials in Security Descriptor".
Here is the environment settings on the m/c before executing globusrun-ws command.
1) GLOBUS_LOCATION=/opt/gtk4
2) The Following environment variables set
a.GRID_SECURITY_DIR=$GLOBUS_LOCATION/etc/grid-security
b.GRIDMAP=$GLOBUS_LOCATION/etc/grid-security/grid-mapfile
c.X509_CERT_DIR=$GLOBUS_LOCATION/share/certificates
3)
The $GLOBUS_LOCATION/etc/globus_wsrf_core/global_security_config.xml entry is as
below.
<securityConfig xmlns="http://www.globus.org">
    <credential>
        <key-file value="etc/grid-security/containerkey.pem"/>
        <cert-file value="etc/grid-security/containercert.pem"/>
    </credential>
<gridmap value="etc/grid-security/grid-mapfile"/>
</securityConfig>
4)Used relative path for gridmap entry in required security-config files for gram 
5) Container was started successfully from /home/globus (non GLOBUS_LOCATION)
6) Executed $GLOBUS_LOCATION/bin/globusrun-ws -submit -f mytest.xml -dbg 

(Note: From the debug messages what i could read, i feel it is failing while
calling getSystemSubject method 
2005-03-16 07:47:45,909 DEBUG security.SecurityManagerImpl
[Thread-9,getSystemSubject:76] [SEC] Error obtaining container credentials )

Server Side logs (Detail logs)
2005-03-16 07:47:45,653 DEBUG utils.PersistenceHelper [Thread-9,store:108] done
storing resource
{http://www.globus.org/namespaces/2004/10/gram/job}ResourceID=49bb2bae-95e7-11d9-a59a-00025589a5dd
2005-03-16 07:47:45,655 DEBUG exec.ManagedExecutableJobHome
[Thread-9,create:266] at add()
2005-03-16 07:47:45,659 DEBUG impl.ResourceSweeper
[Thread-9,scheduleSweeper:373] scheduling sweeper
2005-03-16 07:47:45,661 DEBUG exec.ManagedExecutableJobHome
[Thread-9,create:272] Leaving create()
2005-03-16 07:47:45,663 DEBUG factory.ManagedJobFactoryService
[Thread-9,createManagedJob:281] at find()
2005-03-16 07:47:45,672 DEBUG descriptor.ContainerSecurityConfig
[Thread-9,getConfig:74] initialize called
2005-03-16 07:47:45,673 DEBUG factory.ManagedJobFactoryService
[Thread-9,createManagedJob:297] at subscribe()
2005-03-16 07:47:45,700 DEBUG notification.SubscribeHelper
[Thread-9,subscribe:201] class org.apache.axis.types.URI
2005-03-16 07:47:45,857 DEBUG descriptor.ServiceSecurityConfig
[Thread-9,initialize:82] No security descriptor for: SubscriptionManagerService
2005-03-16 07:47:45,860 DEBUG descriptor.ServiceSecurityConfig
[Thread-9,refresh:170] Refresh called SubscriptionManagerService
2005-03-16 07:47:45,863 DEBUG descriptor.ServiceSecurityConfig
[Thread-9,initialize:82] No security descriptor for: SubscriptionManagerService
2005-03-16 07:47:45,868 DEBUG security.SecurityManagerImpl
[Thread-9,getSystemSubject:70] Get system subject
2005-03-16 07:47:45,891 DEBUG descriptor.ContainerSecurityConfig
[Thread-9,getConfig:74] initialize called
subscribe()
2005-03-16 07:47:45,700 DEBUG notification.SubscribeHelper
[Thread-9,subscribe:201] class org.apache.axis.types.URI
2005-03-16 07:47:45,857 DEBUG descriptor.ServiceSecurityConfig
[Thread-9,initialize:82] No security descriptor for: SubscriptionManagerService
2005-03-16 07:47:45,860 DEBUG descriptor.ServiceSecurityConfig
[Thread-9,refresh:170] Refresh called SubscriptionManagerService
2005-03-16 07:47:45,863 DEBUG descriptor.ServiceSecurityConfig
[Thread-9,initialize:82] No security descriptor for: SubscriptionManagerService
2005-03-16 07:47:45,868 DEBUG security.SecurityManagerImpl
[Thread-9,getSystemSubject:70] Get system subject
2005-03-16 07:47:45,891 DEBUG descriptor.ContainerSecurityConfig
[Thread-9,getConfig:74] initialize called
2005-03-16 07:47:45,893 DEBUG descriptor.ContainerSecurityConfig
[Thread-9,refresh:230] Refreshing container security descriptor
2005-03-16 07:47:45,896 DEBUG descriptor.ContainerSecurityConfig
[Thread-9,initCredentials:136] Init creds called
2005-03-16 07:47:45,899 DEBUG descriptor.ContainerSecurityConfig
[Thread-9,initCredentials:139] Not null, calling load creds
2005-03-16 07:47:45,902 DEBUG descriptor.SecurityConfig
[Thread-9,loadCredentials:287] Loading credential: cert =
'etc/grid-security/containercert.pem' key = 'etc/grid-security/containerkey.pem'
2005-03-16 07:47:45,905 DEBUG descriptor.SecurityConfig
[Thread-9,resolvePath:457] File etc/grid-security/containerkey.pem does not
exist wrt currentdir
2005-03-16 07:47:45,909 DEBUG security.SecurityManagerImpl
[Thread-9,getSystemSubject:76] [SEC] Error obtaining container credentials
. Caused by
org.globus.wsrf.config.ConfigException: Failed to load key file:
"etc/grid-security/containerkey.pem"
        at
org.globus.wsrf.impl.security.descriptor.SecurityConfig.getConfigDir(SecurityConfig.java:443)
        at
org.globus.wsrf.impl.security.descriptor.SecurityConfig.resolvePath(SecurityConfig.java:459)
        at
org.globus.wsrf.impl.security.descriptor.SecurityConfig.loadCredentials(SecurityConfig.java:290)
        at
org.globus.wsrf.impl.security.descriptor.ContainerSecurityConfig.initCredentials(ContainerSecurityConfig.java:141)
        at
org.globus.wsrf.impl.security.descriptor.ContainerSecurityConfig.refresh(ContainerSecurityConfig.java:231)
        at
org.globus.wsrf.impl.security.SecurityManagerImpl.getSystemSubject(SecurityManagerImpl.java:73)
        at
org.globus.wsrf.impl.security.SecurityManagerImpl.getServiceSubject(SecurityManagerImpl.java:124)
        at org.globus.wsrf.jndi.BeanFactory.getObjectInstance(BeanFactory.java:65)
        at org.apache.naming.factory.ResourceFactory.getObjectInstance(Unknown
Source)
        at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:314)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
  at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at javax.naming.InitialContext.lookup(InitialContext.java:361)
        at org.apache.naming.SelectorContext.lookup(Unknown Source)
        at javax.naming.InitialContext.lookup(InitialContext.java:361)
        at org.globus.wsrf.jndi.JNDIUtils.lookup(JNDIUtils.java:227)
        at
org.globus.wsrf.impl.ResourceHomeImpl.initialize(ResourceHomeImpl.java:200)
        at
org.globus.wsrf.impl.notification.SubscriptionHome.initialize(SubscriptionHome.java:45)
        at
org.globus.wsrf.jndi.BasicBeanFactory.getObjectInstance(BasicBeanFactory.java:35)
        at org.globus.wsrf.jndi.BeanFactory.getInstance(BeanFactory.java:110)
        at org.globus.wsrf.jndi.BeanFactory.getObjectInstance(BeanFactory.java:76)
        at org.apache.naming.factory.ResourceFactory.getObjectInstance(Unknown
Source)
        at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:314)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at javax.naming.InitialContext.lookup(InitialContext.java:361)
        at org.apache.naming.SelectorContext.lookup(Unknown Source)
        at javax.naming.InitialContext.lookup(InitialContext.java:361)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)

  at org.apache.naming.SelectorContext.lookup(Unknown Source)
        at javax.naming.InitialContext.lookup(InitialContext.java:361)
        at
org.globus.wsrf.impl.notification.SubscribeHelper.subscribe(SubscribeHelper.java:212)
        at
org.globus.exec.service.factory.ManagedJobFactoryService.createManagedJob(ManagedJobFactoryService.java:299)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled
Code))
        at java.lang.reflect.Method.invoke(Method.java(Compiled Code))
        at
org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:384)
        at
org.globus.axis.providers.RPCProvider.invokeMethodSub(RPCProvider.java:104)
        at
org.globus.axis.providers.PrivilegedInvokeMethodAction.run(PrivilegedInvokeMethodAction.java:39)
        at java.security.AccessController.doPrivileged1(Native Method)
        at
java.security.AccessController.doPrivileged(AccessController.java(Compiled Code))
        at javax.security.auth.Subject.doAs(Subject.java:555)
        at org.globus.gsi.jaas.GlobusSubject.runAs(GlobusSubject.java:49)
        at org.globus.gsi.jaas.JaasSubject.doAs(JaasSubject.java:84)
        at org.globus.axis.providers.RPCProvider.invokeMethod(RPCProvider.java:94)
        at
org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:281)
        at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:319)
        at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
        at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
2005-03-16 07:47:45,915 DEBUG security.SecurityManagerImpl
[Thread-9,getServiceSubject:126] [SEC] Service credentials not configured and
was not able to obtain container credentials.
org.globus.wsrf.security.SecurityException: [SEC] Error obtaining container
credentials; nested exception is:
        org.globus.wsrf.config.ConfigException:  [Caused by: Failed to load key
file: "etc/grid-security/containerkey.pem"]
        at
org.globus.wsrf.impl.security.SecurityManagerImpl.getSystemSubject(SecurityManagerImpl.java:77)
        at
org.globus.wsrf.impl.security.SecurityManagerImpl.getServiceSubject(SecurityManagerImpl.java:124)
        at org.globus.wsrf.jndi.BeanFactory.getObjectInstance(BeanFactory.java:65)
        at org.apache.naming.factory.ResourceFactory.getObjectInstance(Unknown
Source)
        at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:314)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.NamingContext.lookup(Unknown Source)
        at org.apache.naming.SynchronizedContext.lookup(Unknown Source)
------------------
Client Side Logs
Submitting job...
=== REQUEST MESSAGE (length 2161) (time 1110955659.962492000) ===
<ns1:Envelope
xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/"><ns1:Header></ns1:Header><ns2:Body
xmlns:ns2="http://schemas.xmlsoap.org/soap/envelope/"><ns3:createManagedJob
xmlns:ns3="http://www.globus.org/namespaces/2004/10/gram/job"
xmlns:ns4="http://docs.oasis-open.org/wsn/2004/06/wsn-WS-BaseNotification-1.2-draft-01.xsd"
xmlns="http://www.globus.org/namespaces/2004/10/gram/job"><ns3:InitialTerminationTime>2005-03-17T06:47:39Z</ns3:InitialTerminationTime><ns3:JobID>uuid:49bb2bae-95e7-11d9-a59a-00025589a5dd</ns3:JobID><ns4:Subscribe><ns4:ConsumerReference
xmlns:ns5="http://schemas.xmlsoap.org/ws/2004/03/addressing"><ns5:Address>https://lnxwsrf3.boeblingen.de.ibm.com:33463/wsrf/services/NotificationConsumerService</ns5:Address><ns5:ReferenceProperties><ns8:ResourceID
xmlns:ns6="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ns7="http://www.w3.org/2001/XMLSchema"
xmlns:ns8="http://www.globus.org/docs.oasis-open.org/wsn/2004/06/wsn-WS-BaseNotification-1.2-draft-01.wsdl"
ns6:type="ns7:string">49bf7a24-95e7-11d9-a59a-00025589a5dd</ns8:ResourceID></ns5:ReferenceProperties></ns4:ConsumerReference><ns4:TopicExpression
xmlns:ns9="http://www.globus.org/namespaces/2004/10/gram/job/types"
Dialect="http://docs.oasis-open.org/wsn/2004/06/TopicExpression/Simple">ns9:state</ns4:TopicExpression><ns4:UseNotify>true</ns4:UseNotify></ns4:Subscribe><ns10:job
xmlns:ns10="http://www.globus.org/namespaces/2004/10/gram/job/description"><ns10:factoryEndpoint
xmlns:ns5="http://schemas.xmlsoap.org/ws/2004/03/addressing"><ns5:Address>https://localhost:8443/wsrf/services/ManagedJobFactoryService</ns5:Address><ns5:ReferenceProperties><ns3:ResourceID
xmlns:ns6="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ns7="http://www.w3.org/2001/XMLSchema"
ns6:type="ns7:string">Fork</ns3:ResourceID></ns5:ReferenceProperties></ns10:factoryEndpoint><ns10:executable>/bin/echo</ns10:executable><ns10:argument>this
is an example_string </ns10:argument><ns10:argument>Globus was
here</ns10:argument><ns10:stdout>${GLOBUS_USER_HOME}/stdout</ns10:stdout><ns10:stderr>${GLOBUS_USER_HOME}/stderr</ns10:stderr></ns10:job></ns3:createManagedJob></ns2:Body></ns1:Envelope>
----------------------------------------------

=== RESPONSE MESSAGE (length 3668) (time 1110955666.279697000) ===
@
@
@
@
@

----------------------------------------------

=== RESPONSE MESSAGE (length 3668) (time 1110955666.279697000) ===
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/03/addressing"><soapenv:Header><wsa:MessageID
soapenv:mustUnderstand="0">uuid:4dd810d0-95e7-11d9-a434-96867e342961</wsa:MessageID><wsa:Action
soapenv:mustUnderstand="0">http://schemas.xmlsoap.org/ws/2004/03/addressing/fault</wsa:Action><wsa:From
soapenv:mustUnderstand="0"
xmlns:ns12="http://www.globus.org/namespaces/2004/10/gram/job"><wsa:Address>https://localhost:8443/wsrf/services/ManagedJobFactoryService</wsa:Address><wsa:ReferenceProperties><ns12:ResourceID
xmlns:ns12="http://www.globus.org/namespaces/2004/10/gram/job">Fork</ns12:ResourceID></wsa:ReferenceProperties></wsa:From><wsa:RelatesTo
RelationshipType="wsa:Reply"
soapenv:mustUnderstand="0">uuid:4a3cb390-95e7-11d9-a59a-00025589a5dd</wsa:RelatesTo></soapenv:Header><soapenv:Body><soapenv:Fault><faultcode>soapenv:Server.userException</faultcode><faultstring>java.rmi.RemoteException:
Job creation failed.; nested exception is:
        </faultstring><detail><ns1:stackTrace
xmlns:ns1="http://xml.apache.org/axis/">java.rmi.RemoteException: Job creation
failed.; nested exception is:

        at
org.globus.exec.service.factory.ManagedJobFactoryService.createManagedJob(ManagedJobFactoryService.java:315)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled
Code))
        at java.lang.reflect.Method.invoke(Method.java(Compiled Code))
        at
org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:384)
        at
org.globus.axis.providers.RPCProvider.invokeMethodSub(RPCProvider.java:104)
        at
org.globus.axis.providers.PrivilegedInvokeMethodAction.run(PrivilegedInvokeMethodAction.java:39)
        at java.security.AccessController.doPrivileged1(Native Method)
        at
java.security.AccessController.doPrivileged(AccessController.java(Compiled Code))
        at javax.security.auth.Subject.doAs(Subject.java:555)

  at org.globus.gsi.jaas.GlobusSubject.runAs(GlobusSubject.java:49)
        at org.globus.gsi.jaas.JaasSubject.doAs(JaasSubject.java:84)
        at org.globus.axis.providers.RPCProvider.invokeMethod(RPCProvider.java:94)
        at
org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:281)
        at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:319)
        at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
        at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
        at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
        at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:450)
        at org.apache.axis.server.AxisServer.invoke(AxisServer.java:285)
        at org.globus.wsrf.container.ServiceThread.doPost(ServiceThread.java:662)
        at org.globus.wsrf.container.ServiceThread.process(ServiceThread.java:393)
        at
org.globus.wsrf.container.GSIServiceThread.process(GSIServiceThread.java:124)
        at org.globus.wsrf.container.ServiceThread.run(ServiceThread.java:297)
Caused by:
        at
org.globus.wsrf.impl.notification.SubscribeHelper.subscribe(SubscribeHelper.java:217)
        at
org.globus.exec.service.factory.ManagedJobFactoryService.createManagedJob(ManagedJobFactoryService.java:299)
        ... 25 more
</ns1:stackTrace><ns2:hostname
xmlns:ns2="http://xml.apache.org/axis/">lnxwsrf3.boeblingen.de.ibm.com</ns2:hostname></detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>
----------------------------------------------
Failed.
globusrun-ws: Error submitting job
globus_soap_message_module: SOAP Fault
Fault code: soapenv:Server.userException
Fault string: java.rmi.RemoteException: Job creation failed.; nested exception is:
------- Comment #1 From 2005-03-16 10:15:24 -------
As far as I know, this is out of GRAM's control.  Reassigning to core.
------- Comment #2 From 2005-03-30 15:13:04 -------
I was unable to replicate this issue with sample service. Looks like at the 
point of loading the gridmap, the MessageContext assocaited with current thread 
is null and hence unable to get the value for MC_CONFIGPATH. Need to walk 
through GRAM to see what the issue is.
------- Comment #3 From 2005-04-09 10:53:05 -------
I've committed a fix that allows me to execute gram jobs using a container
configured with relative paths and started from a non $GL location.

/Sam