Bug 6415 - GSSAPI symbol clash with kerberos causes problems when dlopening GSI libraries
: GSSAPI symbol clash with kerberos causes problems when dlopening GSI libraries
Status: RESOLVED FIXED
: GSI C
Authorization
: 4.2.1
: NorduGrid Linux
: P3 normal
: 4.2.2
Assigned To:
: http://bugzilla.nordugrid.org/show_bu...
:
:
:
  Show dependency treegraph
 
Reported: 2008-09-25 02:32 by
Modified: 2008-10-22 10:57 (History)


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2008-09-25 02:32:31
The GSSAPI library is deliberatly using the same symbols as other GSSAPI
implementations, so that it can be used in a system that loads different GSSAPI
implementations dynamically at runtime.

However, if a library or application links to the GSSAPI library at compilation
time this causes problems, since the symbols of the API can be resolved to use
any GSSAPI implementation if more than one of the GSSAPI libraries are loaded
in the applications address space.

In the case of ARC0 this is usually not a problem, since you can make sure the
globus GSSAPI implementations is loaded first by putting it first in the
linking order of the application. However, in the ARC1 case the globus GSSAPI
is only used in the plugins, so if the main applications has already loaded a
different GSSAPI implementation this will be used even if the plugin is linked
to the globus GSSAPI library:

[ellert@ellert arc1]$
ARC_PLUGIN_PATH=src/hed/dmc/file/.libs:src/hed/dmc/gridftp/.libs
src/clients/cream2/glitecp std.in gsiftp://grid.tsl.uu.se/storage/ellert/fil -d
FATAL
/tmp/cream_client_test//std.in -> gsiftp://grid.tsl.uu.se/storage/ellert/fil
    0 s:        0,0 kB       0,0 kB/s       0,0 kB/s    . . .       
    0 s:        0,0 kB       0,0 kB/s       0,0 kB/s    . . .       
    0 s:        0,0 kB       0,0 kB/s       0,0 kB/s    . . .       
    0 s:        0,0 kB       0,0 kB/s       0,0 kB/s    . . .       
    0 s:        0,0 kB       0,0 kB/s       0,0 kB/s    . . .       
File moving was not succeeded.

With some more verbose debug (to talkative to quote here in its entirety) you
get the error message: "Unspecified GSS failure. No credentials cache found".
The No credentials cache found error is a kerberos error message from the
kerberos GSSAPI implementation.

You can work around the problem bu LD_PRELOADing the globus GSSAPI
implementation:

 [ellert@ellert arc1]$ LD_LIBRARY_PATH=../../globus-dbg/lib
LD_PRELOAD=../../globus-dbg/lib/libglobus_gssapi_gsi_gcc32dbgpthr.so.0
ARC_PLUGIN_PATH=src/hed/dmc/file/.libs:src/hed/dmc/gridftp/.libs
src/clients/cream2/glitecp std.in gsiftp://grid.tsl.uu.se/storage/ellert/fil -d
FATAL
/tmp/cream_client_test//std.in -> gsiftp://grid.tsl.uu.se/storage/ellert/fil
    0 s:        0,0 kB       0,0 kB/s       0,0 kB/s    . . .       

However, this is not really a workable solution.

The real solution is to add version tags to the GSSAPI symbols in the globus
GSSAPI library. There are at least 2 other GSSAPI implementations in the Fedora
linux distribution and both of these have version tags on their symbols:

[ellert@ellert test]$ cat gssapitest.c
#include <gssapi.h>
int main() {
  gss_release_cred(NULL, NULL);
}

[ellert@ellert test]$ gcc -o gssapitest gssapitest.c -lgssapi_krb5

[ellert@ellert test]$ nm gssapitest | grep gss_release_cred
         U gss_release_cred@@gssapi_krb5_2_MIT

[ellert@ellert test]$ gcc -o gssapitest gssapitest.c -lgssglue

[ellert@ellert test]$ nm gssapitest | grep gss_release_cred
         U gss_release_cred@@libgssapi_CITI_2

Whereas the globus version does not have it:

[ellert@ellert test]$ LD_LIBRARY_PATH=../globus-dbg/lib gcc -o gssapitest
gssapitest.c -L ../globus-dbg/lib -lglobus_gssapi_gsi_gcc32dbgpthr

[ellert@ellert test]$ nm gssapitest | grep gss_release_cred
         U gss_release_cred

The attached patch adds a version tag to the GSSAPI symbols in the globus
GSSAPI library:

[ellert@ellert test]$ LD_LIBRARY_PATH=../globus-dbg2/lib gcc -o gssapitest
gssapitest.c -L ../globus-dbg2/lib -lglobus_gssapi_gsi_gcc32dbgpthr

[ellert@ellert test]$ nm gssapitest | grep gss_release_cred
         U gss_release_cred@@globus_gssapi_gsi

When the rest of globus and ARC1 are compiled against this patched version of
globus GSSAPI it works without the LD_PREOPEN workaround:

[ellert@ellert arc1]$
ARC_PLUGIN_PATH=src/hed/dmc/file/.libs:src/hed/dmc/gridftp/.libs
src/clients/cream2/glitecp std.in gsiftp://grid.tsl.uu.se/storage/ellert/fil -d
FATAL
/tmp/cream_client_test//std.in -> gsiftp://grid.tsl.uu.se/storage/ellert/fil
    0 s:        0,0 kB       0,0 kB/s       0,0 kB/s    . . .

Since it is not possible to attach patches to the Globus bugzilla I refer to
the attachments of the original NorduGrid bugzilla report for the patch.
------- Comment #1 From 2008-09-25 10:14:29 -------
This patch is linker specific (fails to link on machines without GNU ld symbol
versioning), so I'm reluctant to add it to the GT distribution. Can you tell me
more about how this will affect things:

- What versions of the linker support this option?
- Will applications linked against one GSSAPI be able to substitute, at
runtime, a different GSSAPI? 

Also, have you looked at http://grid.ncsa.uiuc.edu/gssapi-mechglue/ from NCSA
which allows an application to link in one GSSAPI which delegates to other
GSSAPI implementations? 

Part of the advantage of using GSSAPI is that an application isn't bound to any
particular security implementation. It's unfortunate that is causing problems
for you in this case. 
------- Comment #2 From 2008-09-25 10:49:21 -------
The other potential solution would be to link your plugin with the static
gssapi library that it requires.
------- Comment #3 From 2008-09-29 15:30:20 -------
(In reply to comment #1)
> This patch is linker specific (fails to link on machines without GNU ld symbol
> versioning), so I'm reluctant to add it to the GT distribution. Can you tell me
> more about how this will affect things:
> 
> - What versions of the linker support this option?
> - Will applications linked against one GSSAPI be able to substitute, at
> runtime, a different GSSAPI? 

There are two ways to use a GSSAPI library. Either you develop against a
specific version of the GSSAPI library using knowledge of specific details in
its implementation and link directly to that specific version. Or your code is
portable enough to use any version of the GSSAPI library and you use dlopen to
load the desired GSSAPI version at runtime (or use libgssglue for the same
effect). The use case you describe - to explicitly link against one version of
the GSSAPI library and at runtime use a different one is broken, and neither
the kerberos nor the libgssglue GSSAPI implementation supports this (since they
both have versioned symbols).

> Also, have you looked at http://grid.ncsa.uiuc.edu/gssapi-mechglue/ from NCSA
> which allows an application to link in one GSSAPI which delegates to other
> GSSAPI implementations? 

Yes I have seen it. Using the libgssglue is an alternative to using dlopen if
your code is portable enough to use any version of the GSSAPI library. The
libgssglue itself does have versioned symbols to make sure that if you link
against it you will also get this GSSAPI version.

> Part of the advantage of using GSSAPI is that an application isn't bound to any
> particular security implementation. It's unfortunate that is causing problems
> for you in this case. 

In this case you should use dlopen or libgssglue and not link against one
GSSAPI library explicitly and expect to be able to use a different one at
runtime. (You can possibly not link against any GSSAPI library at all if you
are allowed to have unresolved symbols at linktime. Not all platforms support
this.)

(In reply to comment #2)
> The other potential solution would be to link your plugin with the static
> gssapi library that it requires.

This does not really work either.

There is an established solution to this problem used by both the kerberos and
the libgssglue GSSAPI libraries. Why would globus try to invent a different
solution, when the established solution works.

The libgssglue source uses -Wl,--version-script always. The kerberos source
uses it for systems matching

*-*-linux* | *-*-gnu* | *-*-k*bsd*-gnu)

If you want I can implement such a restriction in the globus patch.
------- Comment #4 From 2008-09-30 10:01:39 -------
My resistance to this patch is purely practical. The one you proposed doesn't
compile on all of the systems we support. It's unfortunate that you see that as
Globus being against some established solution because of that. My other
comments were meant to make sure that you'd exhausted other more-portable
alternatives. If you have a patch that does work, we would consider it.
------- Comment #5 From 2008-10-15 20:55:02 -------
Hi!

Will this patch work for you?

http://www.grid.tsl.uu.se/repos/globus/fedora/9/info/globus-gssapi-gsi-labels.patch

It uses the same condition for when to use the labelled symbols as the kerberos
gssapi sources.
------- Comment #6 From 2008-10-16 14:16:29 -------
I tried that new patch on a debian linux machine and it didn't enable the
versioning like I'd expected.
------- Comment #7 From 2008-10-17 01:14:55 -------
On my Fedora 9 machine it works. The build log says "-Wl,--version-script
-Wl,gssapi.sym" on the lines where the linking of the library is done.

Also this small check shows that it works:

[ellert@ellert ~]$ cat test.c 
void gss_acquire_cred();

int main() {
  gss_acquire_cred();
  return 0;
}
[ellert@ellert ~]$ gcc -o test test.c -lglobus_gssapi_gsi
[ellert@ellert ~]$ nm test | grep gss_acquire_cred
         U gss_acquire_cred@@globus_gssapi_gsi

Did you rebootstrap? This version of the patch changes configure.in and
Makefile.am without changing configure and Makefile.in. I could provide you
with a patch that includes the propagated changes to configure and Makefile.in
if you prefer.

What is the output of config.guess on your debian machine. On my Fedora 9 it is
"i686-pc-linux-gnu", which matches the test I introduced in the patch.
------- Comment #8 From 2008-10-17 07:14:13 -------
I did a little snooping. It looks like $target_alias is undefined in my build
unless I explicitly pass a --target to the configure script, which is not the
default for gpt builds.
------- Comment #9 From 2008-10-17 07:58:28 -------
OK, now I get it.

I have updated the patch at the same URL.

Now it uses $GLOBUS_HOST that is picked up from globus-build-env-${flavor}.sh
instead - this should work for a pure GPT build as well.
------- Comment #10 From 2008-10-20 11:05:03 -------
Committed the latest patch plus another modification to Makefile.am to add
gssapi.sym to EXTRA_DIST so that it can be built from a source package.