Bug 6348 - globus_config.h inconsistences (esp. GLOBUS_OFF_T) causing failures.
: globus_config.h inconsistences (esp. GLOBUS_OFF_T) causing failures.
Status: RESOLVED FIXED
: Toolkit Internals
globus_core
: 4.0.1
: Open Science Grid (OSG) All
: P3 normal
: ---
Assigned To:
:
:
:
:
  Show dependency treegraph
 
Reported: 2008-09-02 16:11 by
Modified: 2008-10-21 14:29 (History)


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2008-09-02 16:11:14
Hi,

A bug was brought to my attention by EGEE, and I'm pretty sure it's in
globus_core. You can see the original bug report to the VDT in
<http://crt.cs.wisc.edu/Ticket/Display.html?user=guest&pass=guest&id=3864>.
EGEE also has a bug report about it:
<https://savannah.cern.ch/bugs/index.php?40734>.

Here's the deal. When the VDT does a Globus installation of the globus_core
package for gcc32*, the results are different on a 32-bit OS than on a 64-bit
OS. This is causing breakage for EGEE. 

For example, when we install globus_core gcc32 on a 32-bit platform, I see:

#define GLOBUS_OFF_T long long
#define GLOBUS_OFF_T_FORMAT "qd"

However, when we install globus_core gcc32 on a 64-bit platform, we see
something else, which allows smaller offsets:

#define GLOBUS_OFF_T int
#define GLOBUS_OFF_T_FORMAT "d"

I don't fully understand how this ended up leading to failures in the gLite
code, but David Smith provided this statement:

> These are defining
> the printf/scanf format specifiers and storage type for the type that
> globus uses to represent offsets. (including lengths). The 32bit RPM
> build has this defined such that 64 bit offsets can be represented and
> stored, while the 32bit libraries in the 64bit RPM seem to have it
> setup to only be able to represent 32 bit offsets. In itself that
> would limit file lengths to 4GB (or possibly 2GB) - or if an
> application is build with one version and run against the other it
> gives corruption or incorrect lengths.

This is definitely happening in Globus 4.0.7 and one earlier version. (4.0.5?)
I haven't checked Globus 4.0.8. 

Are we right in thinking that these inconsistencies are incorrect? If so, how
do we fix them?

If it matters, we build globus_core in a straightforward way:

gpt-build -nosrc FLAVOR

Thanks for any help. 

-alain

-----------------------------------------------------------------
Alain Roy
Open Science Grid Software Coordinator            roy@cs.wisc.edu
http://opensciencegrid.org                 http://vdt.cs.wisc.edu
------- Comment #1 From 2008-09-02 16:36:47 -------
This is happening with the following globus_core version for sure (and maybe
other versions as well):

<gpt_package_metadata Format_Version="0.02" Name="globus_core" >
<Aging_Version Age="0" Major="4" Minor="30" />
------- Comment #2 From 2008-09-12 09:46:01 -------
I'm rather anxious about this bug. Do you have any suggestions as to how I can
proceed to find a fix?

Thanks!
------- Comment #3 From 2008-09-12 11:06:00 -------
It looks to me like the size of globus_off_t is being set to the size of off_t
as defined in <sys/types.h>.

The test is defined in globus_core's config/acsizes.m4, in LAC_CHECK_SIZEOF,
called for off_t from CHECK_SIZES (also defined in acsizes.m4).

This comment is informational:  I do not know what <sys/types.h> looks like on
various different systems, and I won't speculate on whether a change to
acsizes.m4 is warranted or advisable.  
------- Comment #4 From 2008-09-12 14:23:26 -------
We do not, and have never distributed binary globus_core.  If using a gcc32*
flavored globus_core from one architecture+OS combo is not working on a
different architecture+OS combo, my initial reaction would be, "Don't do that".
 That is, it sounds like you have identified a need to create more globus_core
RPMs than you are already.

Or have I misunderstood the nature of this problem?
------- Comment #5 From 2008-09-18 13:44:44 -------
Alain's reply didn't get in automatically, here it is:

Yes, you've misunderstood it--sorry if I didn't communicate clearly. We don't
distribute binary versions of globus_core, but build it on the platform in
question.

The problem is that the result of building the gcc32 flavor on a 32-bit box is
different from building the gcc32 flavor on a 64-bit box, and in a way that
appears to be incorrect.

-alain
------- Comment #6 From 2008-09-18 13:57:28 -------
In order to detect the correct flags for large file support (LFS), we check the
output of 'getconf LFS_CFLAGS'.  On older distros of Linux, this usually
returns -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 or similar.  Newer distros
seem to have switched to a new posix standard, and now return nothing.  You can
select the standard you want via the -v flag to getconf, so that 'getconf -v
POSIX_V6_ILP32_OFFBIG LFS_CFLAGS' returns what we expect (though it doesn't
seem that all distros retain the old standards so that doesn't work).  

I've only run into this on 2 hosts so I hadn't tried to fix it in core, but
what I did on those hosts is create a wrapper script to getconf that checks if
$1 is LFS_CFLAGS and either calls /usr/bin/getconf -v or simply prints the
common defines above, and have that script ahead of the system getconf in my
path.  Building with that in place works.

I plan to have a real fix in for 4.2.1, so you can expect the 4.0.x one soon as
well.
------- Comment #7 From 2008-09-18 19:37:10 -------
That sounds great! Is it possible to get it as a patch for 4.0.x?

Thanks!
-alain
------- Comment #8 From 2008-09-18 22:14:06 -------
Well its awkward to issue an advisory for core, since it would require a
rebuild of everything else.  A normal patch to the installer shouldn't be a
problem.
------- Comment #9 From 2008-10-20 10:52:15 -------
This was fixed in globus_4_0_branch and was included in the 4.2.1 release.  Do
you need a patch to the 4.0.8 installer or do you generate your own installer
from cvs?  I think the easiest way may just be a core package that you can
untar/overwrite into the installer.

Mike
------- Comment #10 From 2008-10-20 13:28:50 -------
I build from CVS. 

I'm not particular how I patch it, but I would like to be able to fix it in my
upcoming 4.0.8 rebuild of Globus. 

Thanks,
-alain
------- Comment #11 From 2008-10-20 13:56:28 -------
If you're generating the installer from the globus_4_0_branch tag, you'll
already have the fix.  Assuming you're using the globus_4_0_8 tag, untar
http://www.mcs.anl.gov/~mlink/bugs/globus_core-4.35.tar.gz from the
source-trees dir of the installer.
------- Comment #12 From 2008-10-21 12:54:52 -------
Thanks for the tarball!

I distribute GPT separately from Globus, so I assume I should update the core
that is shipped with GPT, right? 

Also, EGEE would really like this built against GT 4.0.3. (They're in the
process of upgrading, but need this bug fix before they can upgrade.) Is there
any reason to be worried about updating globus_core for GT 4.0.3? I don't see
any reason to be worried, but I'm also not an expert in globus_core. 

Thanks,
-alain
------- Comment #13 From 2008-10-21 14:12:31 -------
Yes, you'll want to update the packaging_tools/etc/globus_core-src.tar.gz with
the updated version.
------- Comment #14 From 2008-10-21 14:29:16 -------
It should work fine updating a 4.0.3 installer.