Installing Babel on BG/P (Surveyor at Argonne)

(Notes by Boyana Norris [email protected]…)

I do not recommend that anyone tries this unless they want to spend a lot of time waiting for configure scripts to complete (e.g., on my laptop, babel configure takes 2.5 minutes, while on the BG/P it takes at least 20 minutes, depending on various options, and then you have to actually build).

1. First of all, I started out with the goal of getting as many languages supported as possible, including python. This necessitated the following steps:
   1. Python distutils are rather crippled when it comes to cross-compiling. I built my own python (version 2.6.1) and extended the distutils implementation with a BG/P-specific compiler module. More details on that are available here.
   2. I had to install NumPy , which turned out to be nontrivial (see description). My second requirement was support for shared libraries since fully static builds are (1) less flexible; and (2) not quite fully automated for CCA components. There is also the additional danger of running out of memory because of bloated static executables. GNU Libtool does not work out of the box for building shared libraries on the BG/P -- more on that later.

2. As expected, GNU Autotools-based packages cannot be simply built on BG/P. First, configure does not provide good cross-compilation support, at least not good enough for BG/P. I added these settings to the llnl_cross_compiling.m4 file (right after the case "$target" in line):

     powerpc64-ibm-bgp*)
       cross_compiling=yes
       llnl_cross_compiling_okay=yes
       enable_pure_static_runtime=no
       enable_shared=yes
       enable_static=no
       enable_java=no
       enable_python=/bgsys/drivers/ppcfloor/gnu-linux/bin/python
       llnl_cv_python_frontend=python
       llnl_cv_python_prefix=/bgsys/drivers/ppcfloor/gnu-linux
       llnl_cv_python_numpy=yes
       llnl_cv_python_numerical=no
       llnl_cv_python_library=$llnl_cv_python_prefix/lib
       llnl_cv_python_version=2.5
       llnl_cv_python_include=$llnl_cv_python_prefix/include/python$llnl_cv_python_version
       llnl_cv_extra_python_build_options="--compiler=mpixlc"
       llnl_python_shared_library=$llnl_cv_python_library/libpython2.5.so
       llnl_python_shared_library_found=yes
       sidl_cv_f77_false=0
       sidl_cv_f77_true=1
       sidl_cv_f90_false=0
       sidl_cv_f90_true=1
       llnl_cv_F77_logical_size=4
       llnl_cv_F90_logical_size=4
       ac_cv_f90_pointer_size=8
       llnl_cv_F77_string_passing="far int32_t"
       llnl_cv_F90_string_passing="far int32_t"
       ac_cv_func_malloc_0_nonnull=yes
       ac_cv_func_realloc_0_nonnull=yes
       ac_cv_func_memcmp_working=yes
       with_sidlx=no
   
       ;;
   

Some additional hard-coded configure settings are discussed in this ​IBM Document. (Note: A host triplet used for identifying compute nodes (used by IBM)seems to be powerpc-bgp-linux (not the one above).)

Next I changed the runtime/m4/llnl_confirm_babel_python.m4 file as follows:

*** llnl_confirm_babel_python_support.m4	2008-10-21 12:05:31.000000000 -0500
--- /home/norris/babel-1.4.0/runtime/m4/llnl_confirm_babel_python_support.m4	2009-02-24 22:38:39.246831463 -0600
***************
*** 38,47 ****
--- 38,49 ----
    fi
  
    if test "X$enable_python" != "Xno"; then
+     if test "X$cross_compiling" != "Xyes"; then 
        LLNL_PYTHON_LIBRARY
        LLNL_PYTHON_NUMERIC
        LLNL_PYTHON_SHARED_LIBRARY
        LLNL_PYTHON_AIX
+     fi
      if test \( "X$llnl_cv_python_numerical" != "Xyes" -a  "X$llnl_cv_python_numpy" != "Xyes" \) -o "X$enable_shared" = "Xno" -o "X$XML2_CONFIG" = "Xno"; then
         enable_python=no;
         AC_MSG_WARN([Configuration for Python failed.  Support for Python disabled!])

Finally, I modified runtime/config/config.sub as follows:

*** config.sub	2008-10-21 11:42:24.000000000 -0500
--- /home/norris/babel-1.4.0.bk/runtime/config/config.sub	2009-02-23 13:23:35.376490273 -0600
***************
*** 1391,1396 ****
--- 1391,1399 ----
  	-bgl)
  		os=-bgl
  		;;
+ 	-bgp)
+ 		os=-bgp
+ 		;;
  	-catamount)
  		os=-catamount
  		;;

While building prerequisites (see next item), I found that libtool did not support building and linking shared libraries using IBM's compilers. The perfect fix for this problem is to add BG/P support to libtool. The faster workaround was to first set an environment variable:

export LDFLAGS='-G -qnostaticlink'

then run configure. Next we introduce an extra step before running make by using this little script on all generated libtool scripts in the build directory of the package being built

#!/bin/sh

#File: libtool_fix.sh

wd=`pwd`
if test "x$1" = "x" ; then prog=libtool
else prog=$1; fi

find $wd -name $prog -exec \
    sed -i1 -e 's|^export_dynamic_flag_spec=.*$|export_dynamic_flag_spec=""|g' \
        -e 's|^whole_archive_flag_spec=.$|whole_archive_flag_spec=""|g' \
	-e 's|^pic_flag=.*$|pic_flag=" -DPIC -qpic"|g' \
	-e 's|^archive_cmds=.*$|archive_cmds="\\$CC -G \\$libobjs \\$deplibs \\$compiler_flags -qmkshrobj -e \\$soname  -o \\$lib"|g' \
	-e 's|^hardcode_libdir_flag_spec=.*$|hardcode_libdir_flag_spec="-R\\$libdir -L\\$libdir"|g' {} \; -print

3. To successfully configure Babel, the following prerequisites were necessary: libxml2 and zlib. The former was required for Python support (so the fact that Babel includes libparsifal does not remove the dependence on libxml2 if you want Python). The latter was required by libxml2.
   
   1. zlib-1.2.3 was configured with:

      ./configure --prefix=/home/norris/cca
      

      and then I had to manually change the values of the CC and LDSHARED variables since zlib does not have a real configure. The values were CC=mpixlc and LDSHARED=mpixlc (I think you only really need CC).
      
      
   2. libxml2-2.7.3 was configured and built with

      ./configure --prefix=/home/norris/cca CC=mpixlc CXX=mpixlcxx F90=mpixlf2003 F77=mpixlf77 \
        CPP='gcc -E' --with-python=/bgsys/drivers/ppcfloor/gnu-linux/bin/python --with-zlib=/home/norris/cca \
        --enable-shared --disable-static
      libtool_fix.sh
      make
      make install