Bugzilla – Bug 1651
Intel compiler name change
Last modified: 2004-05-28 11:35:35
You need to log in before you can comment on or make changes to this bug.
I originally submitted this as a bug report against GPT; see <https://bugzilla.ncsa.uiuc.edu/show_bug.cgi?id=233>. Apparently the fix will need to be made in globus_core, not (or not only) in GPT. The following is largely redundant with the description in the GPT bug report, but I've tried to organize it a bit better here. The name of the executable for Intel's Linux C/C++ compiler has changed from version 7 to version 8. (TeraGrid is in the midst of a transition to version 8.) In version 7, the C compiler executable is called "icc" on IA-32 and "ecc" on IA-64. In version 8, it's called "icc" on both IA-32 and IA-64. For compatibility, version 8 on IA-64 provides an "ecc" command, but it prints a warning that can be suppressed with the "-quiet" option. The "-quiet" option is undocumented, and I don't know what other effects it might have; I don't recommend using it. Trying to ignore the warning message is probably also not practical; see bug # 488, where a single warning message from gcc 3.2 caused much gnashing of teeth for many months. Currently, the vendorcc flavor on IA-64 Linux uses "ecc" if it exists. There doesn't seem to be any support for "icc" on IA-32 Linux (a bug that's been reported elsewhere). Though the situation appears complex, I believe the solution is simple. On both IA-32 and IA-64 Linux, the "vendorcc" flavor should use "icc" if it exists; otherwise it should use "ecc". For Intel version 7, this will use "icc" on IA-32 and "ecc" on IA-64 (where there is no "icc" executable). For Intel version 8, this will use "icc" on IA-32 and "icc" on IA-64; both "icc" and "ecc" exist on IA-64, but "icc" is the right command to use. It may also be necessary to pass the "-no-gcc" option to the Intel C compiler. Without this option, the Intel 8 C compiler defines several gcc-specific macros: __GNUC__, __GNUC_MINOR__, and __GNUC_PATCHLEVEL__. (Basically, the Intel compiler's default behavior is to lie and claim to be gcc.) A test Globus build without the "-no-gcc" option failed; adding "-no-gcc" fixed the problem. (I can dig up the details if necessary). The "-no-gcc" option is documented for the Intel version 8 compiler, but not for the version 7 compiler. The version 7 compiler doesn't define the GNUC macros either with or without "-no-gcc", but it does silently ignore the option. I think it's safe to blindly use the "-no-gcc" option for the Intel C compiler, both versions 7 and 8 on both IA-32 and IA-64. Also, the Intel Fortran compiler, called "ifc" (IA-32) or "efc" (IA-64) in version 7, is called "ifort" in version 8. In order to run executables created by the Intel compiler, version 7 or 8, $LD_LIBRARY_PATH has to include the compiler's lib directory. I don't think this is an issue for GPT or Globus; $LD_LIBRARY_PATH should already be set appropriately in the user's environment. I've done a successful test build of Globus 2.4.3 with GPT 3.0.1 using the Intel version 8 compiler on IA-64 Linux. I created an ugly kludge of a compiler setup to fool GPT into thinking it was using the Intel version 7 compiler. I created a $HOME/intel8 directory containing symlinks into the Intel version 8 installation tree -- except that "ifc" is a symlink to "ifort", and "ecc" is a script that invokes "icc" with the "-no-gcc" option. This is not a good approach for a production build; it was intended as a possibility proof. It also demonstrated that there are no hidden problems caused by other incompatibilities in the Intel version 8 compiler (or rather, that any such problems are very well hidden).
Committed a fix for this to CVS HEAD and the 3.2 branch. Not sure how I feel about doing something about this in the 2.4 branch since globus core is not easily updatable.
Gee. That was quick. Do I need to help test this or can Keith handle it?
The TeraGrid needs a 2.4 version of this fix. The globus_core package is included as part of gpt-<VERSION>-src.tar.gz, in packaging_tools/etc/globus_core-src.tar.gz. It's also included in each of the 9 base bundles, as globus_core-2.15.tar.gz. If an updated globus_core package is incorporated into the gpt source tarball, will that override the version that's in the bundles? In other words, will an updated GPT address this problem without having to update the 9 bundles? If so, we'd like to get an updated GPT source tarball that incorporates this fix, preferably via the official ftp://ftp.ncsa.uiuc.edu/aces/gpt/ ftp site (call it 3.1.1b4, I suppose). I can grab the patch and apply it myself for testing if necessary (if someone can tell me how), but I'd feel more comfortable using a GPT tarball built by someone who knows what he's doing. If I'm not making sense, please tell me what I've missed.
Looks like this works on both 32bit and 64bit teragrid machines which means I can close some bugs. Thanks Sam and Keith.
I can do this but I would want Sam or someone to OK releasing a patched globus_core-2.x source package as part of GPT. Is there any reason we can't use the latest globus_core version with the teragrid stuff?
I just tried a Globus test build with GPT 3.1.1b4 (which I understood to be the beta release that addresses the Intel v8 compiler issues). The build failed. The globus_core-2.19 that's included in gpt-3.1.1b4-src.tar.gz looks good (it always searches for "icc" before "ecc", and it appears to apply the "-no-gcc" option correctly). The problem is that the 2.4.3 bundles include a copy of globus_core-2.15. I'm going to try again with gpt-build's "-coresrc" option pointing to a copy of globus_core-2.19 extracted from gpt-3.1.1b4-src.tar.gz, but I'm confused about why I have to do this. The version of globus_core included in GPT is 2.19; shouldn't it ignore the globus_core-2.15 in the bundles? Or should I build the bundles with the "-update" option? What's the best way to do this?
Could you send me a -verbose log. gpt-build is supposed to be only building the latest version of globus_core it finds.
I'll attach the output of gpt-wizard, which includes gpt-build verbose output. The second test build, with the "-coresrc" option, failed in the same way; as far as I can tell the "-coresrc" option was ignored.
Created an attachment (id=352) [details] Build log
Another problem. I kludged together new versions of the Globus bundles, with globus_core-2.15 replaced by globus_core-2.19. (This was intended for testing only; we can use this setup for production builds if we have to, but I'd rather not have to.) It correctly found icc and ifort, but the configure script thinks that "icc" is really "gcc". I'll attach the gpt-wizard log showing the error, along with a fragment of the "configure" script for globus_core-2.19. As far as I can tell, the "configure" script determines whether we are using GNU C by invoking the preprocessor only (icc -E conftest.c). Here's conftest.c: #ifdef __GNUC__ yes; #endif Obviously this isn't a compilable program, but "configure" just checks whether the preprocessor output includes the word "yes". The problem, it doesn't pass the "-no-gcc" option to icc. WARNING: Everything below this line is guesswork and speculation on my part. I don't think this necessarily indicates an actual incompatibilty between icc and gcc. I think that gpt-build, because it thinks the compiler is a GNU C compiler, decides that the vendorcc flavors are not supported. I would guess that the fix is to put the "-no-gcc" option into the compiler name ($CC) rather than into the compiler options. I'm not sure of the implications of a multi-word value for $CC. In other words, in aclocal.m4, change this: AC_PATH_PROGS(lac_cv_CC, $CC icc ecc cc) AC_PATH_PROGS(lac_cv_CXX, $CXX $CCC icc ecc CC c++) AC_PATH_PROGS(lac_cv_F77, $F77 ifort efc f77) AC_PATH_PROGS(lac_cv_F90, $F90 ifort efc f90) # should really check that we really are dealing # with intel compiler lac_CFLAGS="$lac_CFLAGS -no-gcc -restrict" to this (changing just the last line): AC_PATH_PROGS(lac_cv_CC, $CC icc ecc cc) AC_PATH_PROGS(lac_cv_CXX, $CXX $CCC icc ecc CC c++) AC_PATH_PROGS(lac_cv_F77, $F77 ifort efc f77) AC_PATH_PROGS(lac_cv_F90, $F90 ifort efc f90) # should really check that we really are dealing # with intel compiler lac_cv_CC="$lac_cv_CC -no-gcc -restrict" There are probaby other places where this or similar changes would need to be made. The code in question is under "*ia64-*linux*"; the same needs to be done for IA-32 Linux. This should probably affect both lac_cv_CC and lac_cv_CXX. I'll try yet another test build with $CC set to "icc -no-gcc -restrict".
Created an attachment (id=353) [details] Second build log
Created an attachment (id=354) [details] Fragment of configure script for globus_core-2.19
> I'll try yet another test build with $CC set to "icc -no-gcc -restrict". That didn't work either. I won't attach the whole log again (unless somebody wants me to), but the relevant section seems to be: checking for compilers... checking for icc... /usr/local/apps/intel/compiler8/bin/icc checking for icc... /usr/local/apps/intel/compiler8/bin/icc checking for ifort... /usr/local/apps/intel/compiler8/bin/ifort checking for ifort... /usr/local/apps/intel/compiler8/bin/ifort checking whether we are using GNU C... yes checking C flags... -g -no-gcc -restrict -Wall I should be able to work around it by replacing icc with a wrapper script that invokes the real icc with "-no-gcc", but that's ugly.
I changed gpt-build so that it overrides older globus_core pkgs in bundles you can find this version here: ftp://ftp.ncsa.uiuc.edu/aces/gpt/releases/latest-dev/ As for the other problem. GPT checks to make sure that vendorcc flavors are not set to GCC. This is needed on other linux systems so I don't want to remove the check. I'll take a look tomorrow.
Keith, could you attach the core config.log file for the run in comment #9? /Sam
Michael, Couldn't you, in GPT, change the logic in FlavorDefinition.pm to be a little smarter about how it uses the results of the gcc configure test, for example by checking whether it is ecc (or icc) before rejecting the vendorcc flavor?
Due to some laptop issues, I can't post a reply to Bugzilla for now. > Keith, could you attach the core config.log file for the run in comment #9? Sorry, I already nuked the build directory. I've just done a new build that fails for different reasons; I kludged a wrapper for icc that passes the "-no-gcc -restrict" options, so it doesn't exhibit the other problem. I'll do another test build without the wrapper and send you the config.log. Feel free to post this to Bugzilla. -- Keith Thompson, San Diego Supercomputer Center kst@sdsc.edu <http://users.sdsc.edu/~kst/> 858-822-0853 Schroedinger does Shakespeare: "To be *and* not to be"
I just ran: gpt-build vendorcc32 -verbose -nosrc using gpt-3.1.1b5 and it built globus_core-2.19 just fine. How can I reproduce the error that Keith is seeing.
Sandra showed me how to run with the new compilers and I was able to reproduce the ISGCC error. The problem is that AC_PROG_CC_GNU does not use the CFLAGS variable by default which means that the -nogcc flag was never passed in. I added calls to LAC_PROG_CC_GNU for both the ia44 linux and *linux cases and got it to work. I have know idea why the old compilers don't have the same problem. Patch below: =================================================================== RCS file: /home/globdev/CVS/globus-packages/core/source/config/accompiler.m4,v retrieving revision 1.43 diff -r1.43 accompiler.m4 258a259,263 > LAC_PROG_CC_GNU([$lac_cv_CC $lac_CFLAGS], > [ > lac_CFLAGS="$lac_CFLAGS -Wall" > ], > []) 740a746,750 > LAC_PROG_CC_GNU([$lac_cv_CC $lac_CFLAGS], > [ > lac_CFLAGS="$lac_CFLAGS -Wall" > ], > [])
Again, I don't think that is the real problem. Could someone please attach a config.log from a failed configure run? /Sam
Created an attachment (id=358) [details] config.log with new compiler config.log with new compiler
Created an attachment (id=359) [details] gpt-build -verbose output
Created an attachment (id=360) [details] trace of configure run Command used: CC=cc; export GLOBUS_CC; /bin/sh -x /home/mbletzin/Work/gpt-3.2b1/BUILD/globus_core-3.8/BUILD/globus_core-3.8//configure --with-flavor=vendorcc32 > ../trace.log 2>&1
Applied something similar to your patch. Seems to work now. Do you want official core packages for this? /Sam
Yes please
Created an attachment (id=362) [details] Core package for 3.2
Created an attachment (id=363) [details] Core package for 2.4/3.0
That should do it I hope.
I haven't been able to get this to work. I built a patched GPT source tarball by taking a copy of gpt-3.1.1b5-src.tar.gz and replacing gpt-3.1.1b5/packaging_tools/etc/globus_core-src.tar.gz (which is globus_core-2.19) with the new globus_core-2.21 (which as far as I can tell is available only as an attachment to this bug report). I also re-assembled the nine bundles from Globus 2.4.3, replacing the globus_core-2.15.tar.gz package with globus_core-2.21.tar.gz. (This step shouldn't be necessary; I get the same results with the original 2.4.3 bundles from the Globus ftp site.) I set up my environment to use the Intel version 8 compiler (Version 8.0 Build 20040212 Package ID: l_cc_pc_8.0.058_pl060). I unpacked my patched GPT tarball and ran build_gpt, with no problems. I then ran gpt-build -verbose \ globus-information-services-client-2.4.3-src_bundle.tar.gz \ vendorcc64dbgpthr (using the patched bundle mentioned above), and the build failed with "ERROR: vendorcc not supported" while trying to build the globus_core package. It appears that it's still confused by the fact that the Intel compiler pretends to be gcc, but isn't called "gcc". I'll attach the gpt-build output and the config.log file for the globus_core build. In the gpt-build output, I think the problem is: checking whether we are using GNU C... yes checking C flags... -g -no-gcc -restrict -Wall In the config.log output, we can see the following: configure:9032: checking whether we are using GNU C configure:9041: /usr/local/apps/intel/compiler8/bin/icc -E conftest.c Note that it's not passing the "-no-gcc -restrict" options to "icc -E", so, as before, the test concludes that it's using GNU C. I think I can work around this by creating a kludged Intel compiler installation with an "icc" command that invokes the real "icc" with "-no-gcc -restrict", but that's ugly.
Created an attachment (id=367) [details] gpt-build log
Created an attachment (id=368) [details] config.log
I think I have a better workaround for this, one that doesn't require a wrapper script for icc. Details after I confirm that it works.
Yes, it works. I found the following in the ENVIRONMENT VARIABLES section of the icc man page: ICCCFG Specifies the configuration file for customizing compilations with the icc compiler. ICPCCFG Specifies the configuration file for customizing compilations with the icpc compiler. If the $ICCCFG environment variable is set to point to a text file containing the line -no-gcc icc will act as if the "-no-gcc" option had been passed on its command line. It's probably safest to set $ICPCCFG to point to the same file, though I don't think gpt-build will invoke icpc. (The only difference between icc and icpc is that icc assumes preprocessed source files (*.i) are C, and icpc assumes they're C++.) I've added a configuration option to gpt-wizard to do this automatically. This feature will be in the next release, 0.7.4, probably later this week. <http://users.sdsc.edu/~kst/gpt-wizard/> The ideal solution would be for the configuration script not to freak out when a compiler specified by a vendorcc flavor pretends to be gcc, but this should do for now.
Created an attachment (id=374) [details] Core package for 2.4/3.0
Created an attachment (id=375) [details] Core package for 3.2
New core packages that should finally work. The last time I tested I forgot to put the newer intel compilers in my path.
Created an attachment (id=382) [details] Globus Core 3.13 (GT3.2)
Created an attachment (id=383) [details] Globus Core 2.23 (GT2.4)
Ok, completely breaking the GNU C test wasn't quite what I had intended. This update fixes that.
