Bug 4303 - Globus-WS in Tomcat Does Not Honor caCertDir Attribute
: Globus-WS in Tomcat Does Not Honor caCertDir Attribute
Status: RESOLVED FIXED
: CoG jglobus
Tomcat/Axis extensions
: unspecified
: PC Linux
: P3 blocker
: ---
Assigned To:
:
:
:
:
  Show dependency treegraph
 
Reported: 2006-03-21 14:25 by
Modified: 2007-02-05 14:15 (History)


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2006-03-21 14:25:28
In the VDT, users can choose to install CA certificates in a local directory
instead of in /etc/grid-security/certificates/. When Globus is setup to run in
Tomcat, there is an option according to the documentation to specify where this
CA Cert dir is
(http://www.globus.org/toolkit/docs/4.0/common/javawscore/admin-index.html#javawscore-admin-tomcat-deploying).
However, this does not seem to work as Globus always looks in the default
directory regardless of what the configuration variable says.

Steps to reproduce/verify using a VDT installation:

1) Running Globus in Tomcat using default CA Cert Dir
   First we prove that Globus is setup properly and can run jobs in Tomcat
using
   the default configuration.
   Note that /etc/grid-security/certificates/ exists in this test

conf/server.xml:
      <Connector className="org.globus.tomcat.coyote.net.HTTPSConnector"
         port="9443"
         maxThreads="150"
         minSpareThreads="25"
         maxSpareThreads="75"
         autoFlush="true"
         disableUploadTimeout="true"
         scheme="https"
         enableLookups="true"
         acceptCount="10"
         debug="0"
         cert="/etc/grid-security/containercert.pem"
         key="/etc/grid-security/containerkey.pem"/>


11:59am[vdt-fc4-ia32:~]$ globusrun-ws -submit -F localhost:9443 -c /bin/date
Submitting job...Done.
Job ID: uuid:6139b484-b905-11da-bef0-0030482e1f74
Termination time: 03/22/2006 18:06 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.

2) Running Globus in Tomcat Specifying CA Cert Dir
   Now we specify an alternative CA cert dir and submit the same job.
   This test succeeds because it is still looking in the default directory.
   Note that /etc/grid-security/certificates/ exists in this test

conf/server.xml:
      <Connector className="org.globus.tomcat.coyote.net.HTTPSConnector"
         port="9443"
         maxThreads="150"
         minSpareThreads="25"
         maxSpareThreads="75"
         autoFlush="true"
         disableUploadTimeout="true"
         scheme="https"
         enableLookups="true"
         acceptCount="10"
         debug="0"
         cert="/etc/grid-security/containercert.pem"
         key="/etc/grid-security/containerkey.pem"
         cacertdir="/home/pavlo/vdt-1.3.11/globus/TRUSTED_CA"/>

Execute the same command to submit the test job up above, and get the same
successful output

12:09pm[vdt-fc4-ia32:~]$ globusrun-ws -submit -F localhost:9443 -c /bin/date
Submitting job...Done.
Job ID: uuid:6139b484-b905-11da-bef0-0030482e1f74
Termination time: 03/22/2006 18:06 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.

3) Running Globus in Tomcat Specifying CA Cert Dir
   For this test, I remove /etc/grid-security/certificates/ but use the exact
   same configuration for conf/server.xml as I use in #2
   This test proves that it always looks for the certificates in the default
   location no matter what I put as the cacertdir property. I have tried
   different capitalizations of cacertdir and always end up with the same
problem

2:08pm[vdt-fc4-ia32:/etc/grid-security]# mv certificates certificates-ORIG
2:09pm[vdt-fc4-ia32:~]$ globusrun-ws -submit -F localhost:9443 -c /bin/date
Submitting job...Done.
Job ID: uuid:a7ccd186-b916-11da-b90f-0030482e1f74
Termination time: 03/22/2006 20:09 GMT
<< HANG >>
globusrun-ws:
globus_service_engine.c:globus_l_service_engine_session_started_callback:2723:
Session failed to start
globus_xio_system_select.c:globus_l_xio_system_try_read:1134:
System error in read: Connection reset by peer
globus_xio: A system call failed: Connection reset by peer


Appropriate output from logs/catalina.out:

2006-03-21 14:10:12,021 WARN  client.ServiceGroupRegistrationClient
[Thread-5,status:471] Warning: Could not register
https://198.51.254.203:9443/wsrf/services/ManagedJobFactoryService to
servicegroup at https://198.51.254.203:9443/wsrf/services/DefaultIndexService
-- check the URL and that the remote service is up.  Remote exception was ;
nested exception is:
        org.globus.common.ChainedIOException: Authentication failed [Caused by:
Failure unspecified at GSS-API level [Caused by: Unknown CA]]
2006-03-21 14:10:12,182 WARN  client.ServiceGroupRegistrationClient
[Thread-5,status:471] Warning: Could not register
https://198.51.254.203:9443/wsrf/services/ManagedJobFactoryService to
servicegroup at https://198.51.254.203:9443/wsrf/services/DefaultIndexService
-- check the URL and that the remote service is up.  Remote exception was ;
nested exception is:
        org.globus.common.ChainedIOException: Authentication failed [Caused by:
Failure unspecified at GSS-API level [Caused by: Unknown CA]]
2006-03-21 14:10:12,326 WARN  client.ServiceGroupRegistrationClient
[Thread-5,status:471] Warning: Could not register
https://198.51.254.203:9443/wsrf/services/ReliableFileTransferFactoryService to
servicegroup at https://198.51.254.203:9443/wsrf/services/DefaultIndexService
-- check the URL and that the remote service is up.  Remote exception was ;
nested exception is:
        org.globus.common.ChainedIOException: Authentication failed [Caused by:
Failure unspecified at GSS-API level [Caused by: Unknown CA]]

4) Running Globus in Tomcat Specifying CA Cert Dir Using Symlink
   Since it can't find the CA Certificates directory from the attribute, to
make
   sure that something is not wrong with my local TRUSTED_CA directory, I'm
going
   to create a symlink from /etc/grid-security/certificates/ to it
   This proves that Globus is still looking at the default location

2:14pm[vdt-fc4-ia32:/etc/grid-security]# ln -s
/home/pavlo/vdt-1.3.11/globus/TRUSTED_CA certificates
2:15pm[vdt-fc4-ia32:~]$ globusrun-ws -submit -F localhost:9443 -c /bin/date
Submitting job...Done.
Job ID: uuid:7f523628-b917-11da-a6ce-0030482e1f74
Termination time: 03/22/2006 20:15 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.
------- Comment #1 From 2006-03-23 21:25:36 -------
I think the Tomcat connector works ok but I don't think your test is quite
correct. First of all, not only Tomcat will use /etc/grid-security/certificates
by default. The globusrun-ws will try to use that, and the gt4 container
(separate from Tomcat connector) will use that by default. So your test must
isolate Tomcat experiments from everything else. I suggest having
/etc/grid-security/certificats directory, and than changing the Tomcat cacerdir
property to some empty directory. That should cause 'unknown CA' errors from
Tomcat or in Tomcat logs only. 
------- Comment #2 From 2006-04-04 13:14:12 -------
I double checked this and the cacertdir property works fine for Tomcat 4.1.x
and 5.0.x but did not exactly for Tomcat 5.5.x. It works ok for Tomcat 5.5.x if
the property is specified as 'cACertDir' instead of 'cacertdir'. Tomcat 5.5.x
seems to be touchy about the case.
I fixed up the code in cvs to accept the 'cacertdir' property in lowercase for
Tomcat 5.5.x.
------- Comment #3 From 2006-04-04 13:23:10 -------
I will try again tomorrow. We are running Tomcat 5.0.28
------- Comment #4 From 2006-05-08 13:18:30 -------
Sorry for taking so long to get back to you on this.

I took your advice and separate the client from the server to try to isolate
the problem. 

For the server, I have Tomcat 5.0.28 running on Fedora Core 3. The 'cacertdir'
attribute is set to '/home/pavlo/vdt-tomcat/globus/TRUSTED_CA' which contains
all the CA files. The default directory '/etc/grid-security/certificates/' does
not exist on this machine.

My client is running on a separate Fedora Core 4 machine. The default directory
'/etc/grid-security/certificates/' does exist on this machine.

This is what I have in my 'conf/server.xml' file for Tomcat:

      <!-- Globus Container Connector -->
      <Connector className="org.globus.tomcat.coyote.net.HTTPSConnector"
         port="9443"
         maxThreads="150"
         minSpareThreads="25"
         maxSpareThreads="75"
         autoFlush="true"
         disableUploadTimeout="true"
         scheme="https"
         enableLookups="true"
         acceptCount="10"
         debug="0"
         cert="/etc/grid-security/containercert.pem"
         key="/etc/grid-security/containerkey.pem"
         cacertdir="/home/pavlo/vdt-tomcat/globus/TRUSTED_CA"/>

Now when I execute the same test as before, I am still having problems:

CLIENT:

$ globusrun-ws -submit -F vdt-fc3-ia32.cs.wisc.edu:9443 -c /bin/date
Submitting job...Done.
Job ID: uuid:00af4d7a-debe-11da-b813-0030482e1f74
Termination time: 05/09/2006 18:10 GMT
<< HANG >>

SERVER:

INFO: Server startup in 13159 ms
May 8, 2006 1:11:02 PM
org.globus.wsrf.impl.security.authorization.ServiceAuthorizationChain authorize
INFO: Authorized "/DC=org/DC=doegrids/OU=People/CN=Andrew Pavlo 202950" to
invoke "{http://www.globus.org/namespaces/2004/10/gram/job}createManagedJob".
May 8, 2006 1:11:07 PM
org.globus.wsrf.impl.servicegroup.client.ServiceGroupRegistrationClient status
WARNING: Warning: Could not register
https://198.51.254.220:9443/wsrf/services/ManagedJobFactoryService to
servicegroup at https://198.51.254.220:9443/wsrf/services/DefaultIndexService
-- check the URL and that the remote service is up.  Remote exception was ;
nested exception is:
        org.globus.common.ChainedIOException: Authentication failed [Caused by:
Failure unspecified at GSS-API level [Caused by: Unknown CA]]
May 8, 2006 1:11:07 PM
org.globus.wsrf.impl.servicegroup.client.ServiceGroupRegistrationClient status
WARNING: Warning: Could not register
https://198.51.254.220:9443/wsrf/services/ManagedJobFactoryService to
servicegroup at https://198.51.254.220:9443/wsrf/services/DefaultIndexService
-- check the URL and that the remote service is up.  Remote exception was ;
nested exception is:
        org.globus.common.ChainedIOException: Authentication failed [Caused by:
Failure unspecified at GSS-API level [Caused by: Unknown CA]]
May 8, 2006 1:11:07 PM
org.globus.wsrf.impl.servicegroup.client.ServiceGroupRegistrationClient status
WARNING: Warning: Could not register
https://198.51.254.220:9443/wsrf/services/ReliableFileTransferFactoryService to
servicegroup at https://198.51.254.220:9443/wsrf/services/DefaultIndexService
-- check the URL and that the remote service is up.  Remote exception was ;
nested exception is:
        org.globus.common.ChainedIOException: Authentication failed [Caused by:
Failure unspecified at GSS-API level [Caused by: Unknown CA]]

Now if I create the symlink from '/etc/grid-security/certificates/' to
'/home/pavlo/vdt-tomcat/globus/TRUSTED_CA', everything works:

CLIENT:

$ globusrun-ws -submit -F vdt-fc3-ia32.cs.wisc.edu:9443 -c /bin/date
Submitting job...Done.
Job ID: uuid:799b80a0-debe-11da-8247-0030482e1f74
Termination time: 05/09/2006 18:14 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.

SERVER:

May 8, 2006 1:14:24 PM
org.globus.wsrf.impl.security.authorization.ServiceAuthorizationChain authorize
INFO: Authorized "/DC=org/DC=doegrids/OU=People/CN=Andrew Pavlo 202950" to
invoke "{http://www.globus.org/namespaces/2004/10/gram/job}createManagedJob".
May 8, 2006 1:14:28 PM
org.globus.wsrf.impl.security.authorization.ServiceAuthorizationChain authorize
INFO: Authorized "/DC=org/DC=doegrids/OU=People/CN=Andrew Pavlo 202950" to
invoke "{http://wsrf.globus.org/core/notification}destroy".
May 8, 2006 1:14:30 PM
org.globus.wsrf.impl.security.authorization.ServiceAuthorizationChain authorize
INFO: Authorized "/DC=org/DC=doegrids/OU=People/CN=Andrew Pavlo 202950" to
invoke "{http://www.globus.org/namespaces/2004/10/gram/job/exec}destroy".
May 8, 2006 1:14:36 PM org.globus.mds.index.impl.DefaultIndexService
processConfigFile
INFO: Reading default registration configuration from file:
/home/pavlo/vdt-tomcat/tomcat/v5/webapps/wsrf/WEB-INF/etc/globus_wsrf_mds_index/hierarchy.xml

So I think this might prove my original point that the server does not work
correctly if '/etc/grid-security/certificates/' does not exist. But since you
say it works as expected, could you please tell me what I am missing so I can
update the VDT?
------- Comment #5 From 2006-05-08 14:09:11 -------
The cacertdir paramter in the Tomcat connector control the CA certificates for
the connector only. It does not control the CA certificates for all GT
services. That's controlled by a container security descriptor. See security
documentation. Basically, the error you see on the server side or by hanging
client is not caused by misconfigured Tomcat connector but by misconfigured
service. 
Also, if you see your globusrun output, you actaully see that a job was
successfully submitted ("Submitting job...Done."). Therefore, some requests got
through. If you change the cacertdir parameter to some non-existient directory
you will not even see that much. 
------- Comment #6 From 2006-05-08 15:50:05 -------
Subject: Re:  Globus-WS in Tomcat Does Not Honor caCertDir
  Attribute


>The cacertdir paramter in the Tomcat connector control the CA certificates for
>the connector only. It does not control the CA certificates for all GT
>services. That's controlled by a container security descriptor. See security
>documentation.

I looked in the "Writing server-side security descriptor files", and 
it didn't say anything about setting the trusted CA directory. Can 
you point us more directly?

http://www.globus.org/toolkit/docs/4.0/security/authzframe/security_descriptor.html#s-authzframe-server-secdesc-descFile

Thanks,
-alain
------- Comment #7 From 2007-02-05 14:15:43 -------
Apparently the ability to set CA cert dirs from the container security
descriptor is a trunk-only feature, which is why you can't find it in the 4.0
authzframework documents.  Instead, you can set a cacert value in the tomcat
container owner's cog.properties file, or possibly just have X509_CERT_DIR set
in the environment.