<< back >>

Building Babel on the BG/P Surveyor (GNU)

• The next step is building babel. I prefer to build out of the development repository since I can then commit changes back quicker if I find something that needs fixing. I will assume you have an account. If you do not have one contact Tom Epperly at llnl.

• Build without Java since this isn't supported on backend BG/P nodes.

  >>svn co svn+ssh://svn.cca-forum.org/svn/babel/trunk babelDev ; \
  cd babelDev ;\
  ./autotool_rebuild.sh ;\
  ./configure CC=mpicc CXX=mpicxx FC=mpif90 F77=mpif77 --prefix=$BABEL_INSTALL_GNU --host=powerpc-bgp-linux-gnu \
  --without-sidlx \
  --with-gcc=no --enable-python=/bgsys/drivers/ppcfloor/gnu-linux/bin/python --with-libxml2=$LIBXML2_LOC_GNU \
  LDFLAGS=-qnostaticlink --enable-shared --disable-static host_alias=powerpc-bgp-linux-gnu --target=powerpc64-ibm-bgp \
  --with-ltdl-lib=$LIBTOOL_LOCATION_GNU/lib --with-ltdl-include=$LIBTOOL_LOCATION_GNU/include --without-libparsifal \
  PYTHONPATH=/soft/apps/python/python-2.6-cnk-gcc/numpy-1.3.0/lib/python2.6/site-packages CPP="gcc -E" \
  target_alias=powerpc64-gnu-bgp --enable-fortran03=yes --enable-fortran90=yes --enable-fortran77=yes --enable-java=no \
  --with-arch=IBMXL --with-F90-vendor=IBMXL ; \
  ./contrib/bgpSurveyorLibtoolFix.sh ; \
  make ; make install
  

• The build and install should have proceeded without error.

• Since I was having problems with getting regression tests to run, I first tried to get something in examples/hello working:

  >>cd examples/hello
  >>make check
  >>cd ../..
  >>./contrib/bgpSurveyorExampleFix.sh
  >>cd examples/hello/runPython/
  >>qsub --mode script -A cca-tools -t 5 -n 1 ./runPy2C
  

• If it works you should see the following in an output file:

  >>more 198866.output
  Hello world!
  

• Now run:

  >>qsub --mode script -A cca-tools -t 5 -n 1 ./runPy2Cxx
  >>more 198867.output
  Hello world2!
  process group 974 has completed 
  

• Try running something from runCxx:

  >>cd ../runCxx
  >>qsub --mode script -A cca-tools -t 15 -n 1 runAll.sh
  >>more 198868.output
  Hello from C...
  Hello from C++...
  f77 support not enabled
  

And the output is as we expect it. Now I will move on to regression tests (regression/args) to make sure Cxx->Python and Python->Python work.

• All regression tests must be run individually. Note, when you run a test with 1 node we're still charged for using all 64 processors. For this reason, I run one or two tests to make sure babel was compiled correctly but nothing more. If we run into problems down the road we will test that individual babel functionality or if necessary, build an MPI test harness which would allow us to use all 64 processors.

  >>cd regression/output
  >>make check
  >>cd ../args ; make check
  >>cd ../.. ; ./contrib/bgpSurveyorRegressionFix.sh ; cd -
  

• Python -> {C, Cxx, Python}. For this I change directories to runPython and use the runPy2* scripts. I also added the line below to runPy2C.

  PYTHONPATH=/soft/apps/python/python-2.6-cnk-gcc/numpy-1.3.0/lib/python2.6/site-packages:${PYTHONPATH}
  

  • Python -> C

    >>cd ../runPython
    >>qsub --mode script -A cca-tools -t 5 -n 1 ./runPy2C
    198879
    >>tail 198879.output
    COMMENT: (1, 3.14 + 3.14i) == obj.passoutdcomplex()
    RESULT 46 PASS
    PART 47
    COMMENT: (1, 3.14 - 3.14i) == obj.passinoutdcomplex(3.14 + 3.14i)
    RESULT 47 PASS
    PART 48
    COMMENT: (3.14 + 3.14i, 3.14 + 3.14i, 3.14 + 3.14i) = obj.passeverywheredcomplex(3.14 + 3.14i, 3.14 - 3.14i)
    RESULT 48 PASS
    TEST_RESULT PASS
    process group 991 has completed
    

  • Python -> Cxx

    >>cd ../runPython
    >>qsub --mode script -A cca-tools -t 5 -n 1 ./runPy2Cxx
    198880
    >>tail 198880.output
    COMMENT: (1, 3.14 + 3.14i) == obj.passoutdcomplex()
    RESULT 46 PASS
    PART 47
    COMMENT: (1, 3.14 - 3.14i) == obj.passinoutdcomplex(3.14 + 3.14i)
    RESULT 47 PASS
    PART 48
    COMMENT: (3.14 + 3.14i, 3.14 + 3.14i, 3.14 + 3.14i) = obj.passeverywheredcomplex(3.14 + 3.14i, 3.14 - 3.14i)
    RESULT 48 PASS
    TEST_RESULT PASS
    process group 993 has completed
    

  • Python -> Python

    >>cd Py2Py
    >>qsub --mode script -A cca-tools -t 5 -n 1 ./runPy2Py.sh
    198882
    >>tail 198882.output
    PART 46
    COMMENT: (1, 3.14 + 3.14i) == obj.passoutdcomplex()
    RESULT 46 PASS
    PART 47
    COMMENT: (1, 3.14 - 3.14i) == obj.passinoutdcomplex(3.14 + 3.14i)
    RESULT 47 PASS
    PART 48
    COMMENT: (3.14 + 3.14i, 3.14 + 3.14i, 3.14 + 3.14i) = obj.passeverywheredcomplex(3.14 + 3.14i, 3.14 - 3.14i)
    RESULT 48 PASS
    TEST_RESULT PASS
    

• Cxx -> {Cxx, C}. I run the binaries directly out of the .libs directory. The SIDL_DLL_PATH to the library as well as the argument to lt-runAll is changing.
  • Cxx -> Cxx

    >>qsub --mode vn -n 1 -A cca-tools -t 15 \
    --env SIDL_DEBUG_OPEN=1:\
    LD_LIBRARY_PATH="../../../lib/sidlstubs/Cxx/.libs:/bgsys/drivers/ppcfloor/lib64:/dbhome/db2cat/sqllib/lib64:/dbhome/db2cat/sqllib/lib32":\
    SIDL_DLL_PATH="../libCxx/libImpl.scl;../../output/libC/libOutput.scl;../../../runtime/sidlx/libsidlx.scl" \
    /gpfs/home/muszala/ccaToolChain/babelDev/regression/args/runCxx/.libs/lt-runAll C++
    >>tail 198872.output
    COMMENT: obj.passoutdcomplex( out ) == true && out.real() == 3.14 && out.imag() == 3.14
    RESULT 38 PASS
    PART 39
    COMMENT: obj.passinoutdcomplex( inout ) == true && inout.real() == 3.14 && inout.imag() == -3.14
    RESULT 39 PASS
    COMMENT: retval = obj.passeverywheredcomplex( in, out, inout );
    PART 40
    COMMENT: retval.real() == 3.14 && retval.imag() == 3.14 && out.real() == 3.14 && out.imag() == 3.14 && inout.real() == 3.14 && inout.imag() == 3.14
    RESULT 40 PASS
    TEST_RESULT PASS
    

  • Cxx -> C

    >>qsub --mode vn -n 1 -A cca-tools -t 5 \
    --env SIDL_DEBUG_OPEN=1:\
    LD_LIBRARY_PATH="../../../lib/sidlstubs/Cxx/.libs:/bgsys/drivers/ppcfloor/lib64:/dbhome/db2cat/sqllib/lib64:/dbhome/db2cat/sqllib/lib32":\
    SIDL_DLL_PATH="../libC/libImpl.scl;../../output/libC/libOutput.scl;../../../runtime/sidlx/libsidlx.scl" \
    /gpfs/home/muszala/ccaToolChain/babelDev/regression/args/runCxx/.libs/lt-runAll C
    >>q
    198875  muszala  00:15:00  1      queued  None  
    >>tail 198875.output
    COMMENT: obj.passoutdcomplex( out ) == true && out.real() == 3.14 && out.imag() == 3.14
    RESULT 38 PASS
    PART 39
    COMMENT: obj.passinoutdcomplex( inout ) == true && inout.real() == 3.14 && inout.imag() == -3.14
    RESULT 39 PASS
    COMMENT: retval = obj.passeverywheredcomplex( in, out, inout );
    PART 40
    COMMENT: retval.real() == 3.14 && retval.imag() == 3.14 && out.real() == 3.14 && out.imag() == 3.14 && inout.real() == 3.14 && inout.imag() == 3.14
    RESULT 40 PASS
    TEST_RESULT PASS
    

• F90 -> {F90, C}. I run the binaries directly out of the .libs directory. The SIDL_DLL_PATH to the library as well as the argument to runAll is changing.
  • F90 -> F90

    >>qsub --mode vn -n 1 -A cca-tools -t 15 \
    --env SIDL_DEBUG_OPEN=1:\
    LD_LIBRARY_PATH="../../../lib/sidlstubs/F90/.libs:/bgsys/drivers/ppcfloor/lib64:/dbhome/db2cat/sqllib/lib64:/dbhome/db2cat/sqllib/lib32":\
    SIDL_DLL_PATH="../libF90/libImpl.scl;../../output/libC/libOutput.scl;../../../runtime/sidlx/libsidlx.scl" \
    /gpfs/home/muszala/ccaToolChain/babelDev/regression/args/runF90/.libs/lt-runAll F90
    >>q
    203528  muszala  00:15:00  1      queued  None      
    >>tail 203528.output 
    RESULT 36 PASS
    PART 37
    RESULT 37 PASS
    PART 38
    RESULT 38 PASS
    PART 39
    RESULT 39 PASS
    PART 40
    RESULT 40 PASS
    TEST_RESULT PASS
    

  • F90 -> C

    >>qsub --mode vn -n 1 -A cca-tools -t 10 \
    --env SIDL_DEBUG_OPEN=1:\
    LD_LIBRARY_PATH="../../../lib/sidlstubs/F90/.libs:/bgsys/drivers/ppcfloor/lib64:/dbhome/db2cat/sqllib/lib64:/dbhome/db2cat/sqllib/lib32":\
    SIDL_DLL_PATH="../libC/libImpl.scl;../../output/libC/libOutput.scl;../../../runtime/sidlx/libsidlx.scl" \
    /gpfs/home/muszala/ccaToolChain/babelDev/regression/args/runF90/.libs/lt-runAll C
    >>q
    203529  muszala  00:10:00  1      queued  None      
    >>tail  203529.output
    PART 40
    PART 38
    RESULT 40 PASS
    RESULT 38 PASS
    TEST_RESULT PASS
    PART 39
    RESULT 39 PASS
    PART 40
    RESULT 40 PASS
    TEST_RESULT PASS
    

• If you do run 'make check' you will build the tests, attempt to run them on the login nodes and all will be reported as broken. If you do run 'make check' to completion you will see the following:

  >>make check
  ...
  ...
  305 B struct/runPython/Py2Py/runPy2Py.sh              0   0% [parse error] 
  306 B sort/runPython/Py2Py/runPy2Py.sh                0   0% [parse error] 
  307 B strings/runPython/Py2Py/runPy2Py.sh             0   0% [parse error] 
  308 B wrapper/runPython/Py2Py/runPy2Py.sh             0   0% [parse error] 
  309 B wrapper/runPython/runPy2C                       0   0% [parse error] 
  310 B wrapper/runPython/runPy2Cxx                     0   0% [parse error] 
  Error: gantlet: 3 failed tests and 221 broken tests.
  make[4]: *** [run-gantlet] Error 224
  make[4]: Leaving directory `/gpfs/home/muszala/ccaToolChain/babelDev/regression'
  make[3]: *** [check-local] Error 2
  make[3]: Leaving directory `/gpfs/home/muszala/ccaToolChain/babelDev/regression'
  make[2]: *** [check-am] Error 2
  make[2]: Leaving directory `/gpfs/home/muszala/ccaToolChain/babelDev/regression'
  make[1]: *** [check-recursive] Error 1
  make[1]: Leaving directory `/gpfs/home/muszala/ccaToolChain/babelDev/regression'
  make: *** [check-recursive] Error 1