*** Bug 4744 has been marked as a duplicate of this bug. ***

I got the GSI code run on top of OpenSSL-0.9.8c. I committed the changes to
openssl_upgrade branch. grid-proxy-init -debug -verify succeeds. I also did a
test GridFTP transfer with globus-url-copy. I also ran the gssapi and proxy
tests. It looks good.

Would it be possible to make the new packages work with older (< 0.9.8)
versions
of openssl possibly by using constructs such as:

#if OPENSSL_VERSION_NUMBER .....

We are currently using the "system" openssl rather than the globus_openssl 
package on various systems.

Anders

Anders -

How are you including the system openssl into the build?   If you're not
compiling globus_openssl at all in the first place, we should be able to
provide the ifdef workaround you're looking for.  The code changes to support
the 0.9.7 -> 0.9.8 API changes have been minimal as far as I understand it.

On the other hand, If you're just replacing libraries/headers after the fact,
I'm not sure your suggested workaround would do anything, because the libraries
would already be linked against 0.9.8 openssl libraries, and it's not clear
that replacing the 0.9.8 libraries with links to 0.9.7 libraries would work.

First of all I've separate RPMs for all the needed Globus packages with correct
build and runtime dependencies. For openssl as well as a few others that are
not native Globus packages (eg. openldap, libxml2, xmlsec1, ...) I'm using
virtual RPM packages named <package>-globus (eg. openssl-globus and 
openssl-globus-devel) which makes GPT (and RPM) happy. So in the present case
my openssl-globus virtual package provides "globus_openssl" and have the
following properties:

$ rpm -ql openssl-globus
/opt/globus/etc/gpt/packages/globus_openssl
/opt/globus/etc/gpt/packages/globus_openssl/pkg_data_gcc32pthr_pgm.gpt
/opt/globus/etc/gpt/packages/globus_openssl/pkg_data_gcc32pthr_rtl.gpt
/opt/globus/etc/gpt/packages/globus_openssl/pkg_data_noflavor_data.gpt
/opt/globus/etc/gpt/packages/globus_openssl/pkg_data_noflavor_doc.gpt
$ rpm -q --requires  openssl-globus
openssl  
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
$ rpm -q --provides  openssl-globus
globus_openssl  
openssl-globus = 1.16-0.4

Since I'm using the same src.rpm's for all systems (FC*, RH*, SuSE*, MDK*) I
would appreciate patches for the various Globus packages that works for various
versions of OpenSSL and not just 0.9.8. I suspect that some of the changes in
openssl_upgrade like the call to SSL_library_init would probably be good on any
OpenSSL version.

Anders

I committed some changes that would allow GSI to run on both 0.9.7 and 0.9.8
versions. Anders: If you can check this to make sure it works for you, that
would be great.

Thanks a lot! I'll take a look at this asap.

Anders

Hi

Sorry for the delay. I did a test immediately after your post and the first
tests failed so I had to do some tests. In short: the patches do not work
with OpenSSL 0.9.8. I've tested the patches on versions below 0.9.8 however
with great success. This include the i386 platforms (with latest supported
updates):

Distribution:          OpenSSL version

Red Hat Linux 7.3      0.9.6b
Red Hat Linux 9        0.9.7a
Red Hat Enterprise 3   0.9.7a  (also works on x86_64)
Red Hat Enterprise 4   0.9.7a  (also works on x86_64)
Fedora Core 1          0.9.7a
Fedora Core 2          0.9.7a
Fedora Core 3          0.9.7a
Fedora Core 4          0.9.7f
Mandrake 10.0          0.9.7c
Mandrake 10.1          0.9.7d
Mandrake 10.2          0.9.7e
SuSE  9.2              0.9.7d
SuSE 10.0              0.9.7g
Debian 3.1             0.9.7e

The platforms which does not work are those with OpenSSL > 0.9.8:

Distribution:   OpenSSL version

Fedora Core 5   0.9.8a
Fedora Core 6   0.9.8b
Ubuntu 6.06     0.9.8a

The OpenSSL version are those reported by 'openssl version', but is probably
patched by the distributions.

My tests are simple client globus-url-copy commands against the NorduGrid/ARC
gridftp server. The details of the failed tests are interesting in the sense
that the errors depend on the proxy type. I'll only post the relevant part of
the debug output from the globus-url-copy command. I've tried both rfc and
pre-rfc proxies (below):

== Fedora Core 5 ==
debug: reading into data buffer 0xb600b008, maximum length 1048576
debug: data callback, error globus_xio_gsi: gss_init_sec_context failed.,
buffer  0xb600b008, length 0, offset=0, eof=true
debug: response from gsiftp://interop.dcgc.dk/storage/runpov.sh:
426 Transfer terminated.

debug: operation complete

error: globus_xio_gsi: gss_init_sec_context failed.
OpenSSL Error: s3_clnt.c:894: in library: SSL routines, function
SSL3_GET_SERVER_CERTIFICATE: certificate verify failed
globus_gsi_callback_module: Could not verify credential
globus_gsi_callback_module: Could not verify credential: unsupported
certificate purpose

== Fedora Core 6 ==

debug: reading into data buffer 0xb5fca008, maximum length 1048576
debug: data callback, error globus_xio_gsi: gss_init_sec_context failed.,
buffer 0xb5fca008, length 0, offset=0, eof=true
debug: response from gsiftp://interop.dcgc.dk/storage/runpov.sh:
426 Transfer terminated.

debug: operation complete

error: globus_xio_gsi: gss_init_sec_context failed.
OpenSSL Error: s3_clnt.c:894: in library: SSL routines, function
SSL3_GET_SERVER_CERTIFICATE: certificate verify failed
globus_gsi_callback_module: Could not verify credential
globus_gsi_callback_module: Could not verify credential: unsupported
certificate purpose

== Ubuntu 6.06 ==

debug: reading into data buffer 0xb5a30008, maximum length 1048576
debug: data callback, error globus_xio_gsi: gss_init_sec_context failed.,
buffer 0xb5a30008, length 0, offse
t=0, eof=true 
debug: response from gsiftp://interop.dcgc.dk/storage/runpov.sh:
426 Transfer terminated.

debug: operation complete

error: globus_xio_gsi: gss_init_sec_context failed.
OpenSSL Error: s3_clnt.c:894: in library: SSL routines, function
SSL3_GET_SERVER_CERTIFICATE: certificate ve
rify failed
globus_gsi_callback_module: Could not verify credential
globus_gsi_callback_module: Could not verify credential: unsupported
certificate purpose

I can do more tests if needed. For the record the patch I'm using was taking
from the 4.0 branch (to get the globus_libc_umask changes plus the relevant
patches from the openssl_upgrade branch. The patch can be found from:

http://hep.nbi.dk/~waananen/openssl_upgrade.patch

Cheers

Anders

Was the GridFTP server you were using also using openssl 0.9.8x, or were you
going against a different version?  We have a GPT-packaged 0.9.8d (unpatched,
just wrapped to install with GPT) which we tested grid-proxy-init -verify
-debug and globus-url-copy against, and both worked for us.  I'm wondering if
the GridFTP server version is the source of the discrepancy, or if it is a
0.9.8{a,b} versus 0.9.8d discrepancy that we're seeing.

The server is OpenSSL 0.9.7x. I hope that the clients compiled against 0.9.8x
will work against 0.9.7x.

Anders

That's definitely a requirement.  I wanted to make sure that we were going to
try reproducing your problem in the right environment, which is why I asked. 
We will test it out and report back on this bug.

I ran two tests, between a stock GT4.0.1 installation and the updated openssl
installation.  Both directions of the test were successful.  I did
client<->server each way:

server$ grid-proxy-init
server$ sbin/globus-gridftp-server -p 8181

client$ grid-proxy-init
client$ globus-url-copy -ss "`grid-proxy-info -identity`"
gsiftp://server:8181/etc/group file:///tmp/woowoo

Each machine was able to start a server and have the other end copy a file
over.  I've put up an installer at
http://www-unix.mcs.anl.gov/~bacon/updated_openssl_installer.tar.gz that you
can try to replicate the build I used.  It is globus_4_0_branch + our modified
GSI packages to accomodate the updated openssl (0.9.8d).  I've used the
installer once already, so if you hit any build failures in GSI packages
related to installing into /sandbox/bacon/updated_openssl you can go to the
srcdir referenced and just run "make clean".

Assuming your installation of this installer works as expected, we can move on
to finding out why the build you expected would work did not.

Adding a 4.2 milestone.  I'm assuming this will get finished by building
against the system openssl libraries, rather than just updating our included
version.

Comment from a colleague of mine:

(In reply to comment #12)
> > I ran two tests, between a stock GT4.0.1 installation and the updated penssl
> > installation.  Both directions of the test were successful.  I did
> > client<->server each way:
> > 
> > server$ grid-proxy-init
> > server$ sbin/globus-gridftp-server -p 8181
> > 
> > client$ grid-proxy-init
> > client$ globus-url-copy -ss "`grid-proxy-info -identity`"
> > gsiftp://server:8181/etc/group file:///tmp/woowoo

Some additional info.
After having added some debug output to
globus_gsi_callback_X509_verify_cert which simply prints the
subject_name of the context->cert
i get the following output talking to a 0.9.7 based globus gridftpd

globus-url-copy gsiftp://ican.hpc2n.umu.se:54321/etc/hosts
file:///scratch/f4
globus_gsi_callback_X509_verify_cert: s_n
= /O=Grid/O=NorduGrid/CN=host/ican-i.hpc2n.umu.se, depth = 9, purpose = 2
globus_gsi_callback_X509_verify_cert: s_n =
/O=Grid/O=NorduGrid/OU=hpc2n.umu.se/CN=Ake
Sandgren/CN=988424655/CN=276348398, depth = 9, purpose = 2

The second call to X509_verify_cert is the one causing the "unsupported
certificate purpose" problem between 0.9.8 based clients and 0.9.7 based
servers. A 0.9.8 based server doesn't return the second cert.

Talking to a dCache based gridftpd i get
globus-url-copy gsiftp://srm2.ndgf.org/test.0911.2 file:///scratch/f1
globus_gsi_callback_X509_verify_cert: s_n
= /O=Grid/O=NorduGrid/CN=host/srm2.ndgf.org, depth = 9, purpose = 2

and nothing more.

Hence no "purpose" error.

Why does 0.9.7 based globus-gridftp return the second cert (my original
proxy) to the client? And should it be fixed by adding a case for
X509_V_ERR_INVALID_PURPOSE in globus_i_gsi_callback_cred_verify just
like the case for X509_V_ERR_INVALID_CA?

Hope this gives you guys enough clues to come up with a official fix.

Hi,

Has anybody tried this with old style proxies?

Cheers,
Joni Hahkala

Anders -

Can you check the output of "openssl x509 -in usercert.pem -noout -text" for
your usercert?  We've done some tests, and for a user whose cert has the cert
types:
Netscape Cert Type:
SSL Client, SSL Server, S/MIME

it works to run a server with 0.9.7 and a client with 0.9.8.

However for another user with cert types:
Netscape Cert Type: 
SSL Client, S/MIME

it does not.  I'd just like to verify that your failing cert does not have the
"SSL Server" Netscape Cert Type.

(In reply to comment #16)
> Anders -
> 
> Can you check the output of "openssl x509 -in usercert.pem -noout -text" for
> your usercert?  We've done some tests, and for a user whose cert has the cert
> types:
> Netscape Cert Type:
> SSL Client, SSL Server, S/MIME
> 
> it works to run a server with 0.9.7 and a client with 0.9.8.
> 
> However for another user with cert types:
> Netscape Cert Type: 
> SSL Client, S/MIME
> 
> it does not.  I'd just like to verify that your failing cert does not have the
> "SSL Server" Netscape Cert Type.

He is probably lacking "SSL Server", I am and i have the same problem.
Although the problem we have is also 0.9.7 client vs 0.9.8 server.
Comment #14 originally came from me.

Reg 0.9.7 based clients vs 0.9.8 based servers.
The problem that i have seen here is when the 0.9.7 based client uses a
voms-proxy.

Can you run "openssl x509 -in usercert.pem -text" on the certificate you're
using to get a VOMS proxy from, and look under the "X509v3 extensions:"
section.  What appears under "Netscape Cert Type:" for the original
certificate?  If we're right about what's happening, SSL Server will not be
listed.

Added a workaround to the 0.9.7 incompatibility to the openssl_upgrade branch.

joe

The new openssl handling code has been committed to the trunk. I wrote up some
instructions for using the new code with a GT4 source installer to deploy
without building OpenSSL and put them on the dev.globus wiki:
http://dev.globus.org/wiki/C_Security:_Vendor_OpenSSL

Joe

Hi

I finally had some time to look at this issue. It seems indeed to work now. I
did add a line to globus_openssl.c from globus_openssl_module:

@@ -70,6 +70,7 @@
     X509V3_EXT_METHOD *                 pci_x509v3_ext_meth = NULL;
     X509V3_EXT_METHOD *                 pci_old_x509v3_ext_meth = NULL;

+    SSL_library_init();
     globus_module_activate(GLOBUS_COMMON_MODULE);
     globus_module_activate(GLOBUS_GSI_OPENSSL_ERROR_MODULE);
     mutex_pool = malloc(CRYPTO_num_locks() * sizeof(globus_mutex_t));

in order to load the cryptos.

Cheers

Anders