Bugzilla – Bug 6348
globus_config.h inconsistences (esp. GLOBUS_OFF_T) causing failures.
Last modified: 2008-10-21 14:29:16
You need to log in before you can comment on or make changes to this bug.
Hi, A bug was brought to my attention by EGEE, and I'm pretty sure it's in globus_core. You can see the original bug report to the VDT in <http://crt.cs.wisc.edu/Ticket/Display.html?user=guest&pass=guest&id=3864>. EGEE also has a bug report about it: <https://savannah.cern.ch/bugs/index.php?40734>. Here's the deal. When the VDT does a Globus installation of the globus_core package for gcc32*, the results are different on a 32-bit OS than on a 64-bit OS. This is causing breakage for EGEE. For example, when we install globus_core gcc32 on a 32-bit platform, I see: #define GLOBUS_OFF_T long long #define GLOBUS_OFF_T_FORMAT "qd" However, when we install globus_core gcc32 on a 64-bit platform, we see something else, which allows smaller offsets: #define GLOBUS_OFF_T int #define GLOBUS_OFF_T_FORMAT "d" I don't fully understand how this ended up leading to failures in the gLite code, but David Smith provided this statement: > These are defining > the printf/scanf format specifiers and storage type for the type that > globus uses to represent offsets. (including lengths). The 32bit RPM > build has this defined such that 64 bit offsets can be represented and > stored, while the 32bit libraries in the 64bit RPM seem to have it > setup to only be able to represent 32 bit offsets. In itself that > would limit file lengths to 4GB (or possibly 2GB) - or if an > application is build with one version and run against the other it > gives corruption or incorrect lengths. This is definitely happening in Globus 4.0.7 and one earlier version. (4.0.5?) I haven't checked Globus 4.0.8. Are we right in thinking that these inconsistencies are incorrect? If so, how do we fix them? If it matters, we build globus_core in a straightforward way: gpt-build -nosrc FLAVOR Thanks for any help. -alain ----------------------------------------------------------------- Alain Roy Open Science Grid Software Coordinator roy@cs.wisc.edu http://opensciencegrid.org http://vdt.cs.wisc.edu
This is happening with the following globus_core version for sure (and maybe other versions as well): <gpt_package_metadata Format_Version="0.02" Name="globus_core" > <Aging_Version Age="0" Major="4" Minor="30" />
I'm rather anxious about this bug. Do you have any suggestions as to how I can proceed to find a fix? Thanks!
It looks to me like the size of globus_off_t is being set to the size of off_t as defined in <sys/types.h>. The test is defined in globus_core's config/acsizes.m4, in LAC_CHECK_SIZEOF, called for off_t from CHECK_SIZES (also defined in acsizes.m4). This comment is informational: I do not know what <sys/types.h> looks like on various different systems, and I won't speculate on whether a change to acsizes.m4 is warranted or advisable.
We do not, and have never distributed binary globus_core. If using a gcc32* flavored globus_core from one architecture+OS combo is not working on a different architecture+OS combo, my initial reaction would be, "Don't do that". That is, it sounds like you have identified a need to create more globus_core RPMs than you are already. Or have I misunderstood the nature of this problem?
Alain's reply didn't get in automatically, here it is: Yes, you've misunderstood it--sorry if I didn't communicate clearly. We don't distribute binary versions of globus_core, but build it on the platform in question. The problem is that the result of building the gcc32 flavor on a 32-bit box is different from building the gcc32 flavor on a 64-bit box, and in a way that appears to be incorrect. -alain
In order to detect the correct flags for large file support (LFS), we check the output of 'getconf LFS_CFLAGS'. On older distros of Linux, this usually returns -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 or similar. Newer distros seem to have switched to a new posix standard, and now return nothing. You can select the standard you want via the -v flag to getconf, so that 'getconf -v POSIX_V6_ILP32_OFFBIG LFS_CFLAGS' returns what we expect (though it doesn't seem that all distros retain the old standards so that doesn't work). I've only run into this on 2 hosts so I hadn't tried to fix it in core, but what I did on those hosts is create a wrapper script to getconf that checks if $1 is LFS_CFLAGS and either calls /usr/bin/getconf -v or simply prints the common defines above, and have that script ahead of the system getconf in my path. Building with that in place works. I plan to have a real fix in for 4.2.1, so you can expect the 4.0.x one soon as well.
That sounds great! Is it possible to get it as a patch for 4.0.x? Thanks! -alain
Well its awkward to issue an advisory for core, since it would require a rebuild of everything else. A normal patch to the installer shouldn't be a problem.
This was fixed in globus_4_0_branch and was included in the 4.2.1 release. Do you need a patch to the 4.0.8 installer or do you generate your own installer from cvs? I think the easiest way may just be a core package that you can untar/overwrite into the installer. Mike
I build from CVS. I'm not particular how I patch it, but I would like to be able to fix it in my upcoming 4.0.8 rebuild of Globus. Thanks, -alain
If you're generating the installer from the globus_4_0_branch tag, you'll already have the fix. Assuming you're using the globus_4_0_8 tag, untar http://www.mcs.anl.gov/~mlink/bugs/globus_core-4.35.tar.gz from the source-trees dir of the installer.
Thanks for the tarball! I distribute GPT separately from Globus, so I assume I should update the core that is shipped with GPT, right? Also, EGEE would really like this built against GT 4.0.3. (They're in the process of upgrading, but need this bug fix before they can upgrade.) Is there any reason to be worried about updating globus_core for GT 4.0.3? I don't see any reason to be worried, but I'm also not an expert in globus_core. Thanks, -alain
Yes, you'll want to update the packaging_tools/etc/globus_core-src.tar.gz with the updated version.
It should work fine updating a 4.0.3 installer.