MPICH2 with Boost

[Kraus Philipp <philipp.kraus@…>]

Hi,

I try to compile my first C++ app with MPICH2 (I have been working for
some month with OpenMPI ). I had compiled Boost (1.39) with the "using
mpi / --with-mpi" option.

Under my OSX shell I had set the library pathes:

LIBRARY_PATH=/Developer/SelfExtensions/Qt/4.5.1/lib:/Developer/
SelfExtensions/MPICH/1.1/lib:/Developer/SelfExtensions/GSL/1.9.0/lib:/
Developer/SelfExtensions/Boost/1.39.0-mpich/lib:

LD_LIBRARY_PATH=/Developer/SelfExtensions/Qt/4.5.1/lib:/Developer/
SelfExtensions/MPICH/1.1/lib:/Developer/SelfExtensions/GSL/1.9.0/lib:/
Developer/SelfExtensions/Boost/1.39.0-mpich/lib:

PATH=/Developer/SelfExtensions/Qt/4.5.1/bin:/Developer/SelfExtensions/
MPICH/1.1/bin:/Developer/SelfExtensions/Ant/bin:/usr/bin:/bin:/usr/
sbin:/sbin:/usr/local/bin:/usr/texbin:/usr/X11/bin:/usr/local/bin:/usr/
texbin

DYLD_LIBRARY_PATH=/Developer/SelfExtensions/Qt/4.5.1/lib:/Developer/
SelfExtensions/MPICH/1.1/lib:/Developer/SelfExtensions/GSL/1.9.0/lib:/
Developer/SelfExtensions/Boost/1.39.0-mpich/lib:

OMPI_LDFLAGS=-lboost_mpi-xgcc40-mt-1_39  -lboost_serialization-xgcc40-
mt-1_39

My testcode shows like:

#include <boost/mpi/environment.hpp>
#include <boost/mpi/communicator.hpp>
#include <iostream>
namespace mpi = boost::mpi;

int main(int argc, char* argv[])
{
   mpi::environment env(argc, argv);
   mpi::communicator world;
   std::cout << "I am process " << world.rank() << " of " <<
world.size()
             << "." << std::endl;
   return 0;
}

This code with OpenMPI runs correct, with MPICH2 I get an running error

[MacMini:55917] *** Process received signal ***
[MacMini:55917] Signal: Segmentation fault (11)
[MacMini:55917] Signal code: Address not mapped (1)
[MacMini:55917] Failing at address: 0x44000088
[ 1] [0xbffff0c8, 0x44000088] (-P-)
[ 2] (_ZN5boost3mpi11environmentC2ERiRPPcb + 0x4b) [0xbffff128,
0x00005ddb]
[ 3] (main + 0x2f) [0xbffff168, 0x00002301]
[ 4] (start + 0x36) [0xbffff180, 0x000022a6]
[ 5] [0x00000000, 0x00000001] (FP-)
[MacMini:55917] *** End of error message ***
Segmentation fault

I stared the mpd without options. With OpenMPI I can run the program
on 2 physical CPU with 4 cores on each CPU, so I get: I am process 0-7
of 8

Can anyone help me to solve my problem? I would like to change from
OpenMPI to MPICH, because I need parallel algorithms under Windows and
Unix Systems

Thanks a lot

Phil

[Kraus Philipp]

Added by email2trac

[Kraus Philipp]

This message has 1 attachment(s)

[balaji]

Phil,

I just tried out Boost-1.39 and it seems to work OK for me on Linux (Ubuntu Jaunty). I don't have a MAC handy to try it out there, but I wanted to point out a few gotcha's nevertheless, in case you missed any:

1. Boost MPI is not very good at overwriting libraries, so if you have had a Boost library compiled with OpenMPI still in your library path, you should clean that first before trying it out with MPICH2.

2. Boost assumes that the MPI C++ wrapper is called mpic++. MPICH2 uses the wrapper mpicxx instead, so auto-detection fails. You can either manually specify the MPI libraries, or just create a symbolic link from mpicxx to mpic++ (we'll be doing this in the next release anyway).

3. OMPI_LDFLAGS are OpenMPI specific flags. If you want to pass these flags to MPICH2's mpicc, you can use:

mpicc [remaining_arguments] ${OMPI_LDFLAGS}

Let us know if this helps.

-- Pavan

[Kraus Philipp]

Hi,

thanks a lot, now it works.


Am 20.06.2009 um 18:14 schrieb mpich2:

> I just tried out Boost-1.39 and it seems to work OK for me on Linux
> (Ubuntu Jaunty). I don't have a MAC handy to try it out there, but I
> wanted to point out a few gotcha's nevertheless, in case you missed
> any:
>
> 1. Boost MPI is not very good at overwriting libraries, so if you
> have had
> a Boost library compiled with OpenMPI still in your library path, you
> should clean that first before trying it out with MPICH2.

I had compiled Boost with this http://www.boost.org/doc/libs/1_38_0/doc/html/mpi
/getting_started.html#mpi.config
steps and set the compilere manually to "using mpi : <path to MPICH-
bin>/mpicc ;"

>
> 2. Boost assumes that the MPI C++ wrapper is called mpic++. MPICH2
> uses
> the wrapper mpicxx instead, so auto-detection fails. You can either
> manually specify the MPI libraries, or just create a symbolic link
> from
> mpicxx to mpic++ (we'll be doing this in the next release anyway).

I had set the env pathes to:

LIBRARY_PATH=< path to MPICH>/lib
LD_LIBRARY_PATH=< path to MPICH>/lib
DYLD_LIBRARY_PATH=< path to MPICH>/lib
CPATH_PATH=< path to MPICH>/include

My sources will be compiled with
mpicxx -o <target> <source> -lboost_mpi-xgcc40-mt-1_39  -
lboost_serialization-xgcc40-mt-1_39

Is there any solution to set the options "-lboost_mpi-xgcc40-mt-1_39  -
lboost_serialization-xgcc40-mt-1_39" to environmental variables like
the OMPI_LDFLAGS?

Thanks for helping

Phil

[balaji]

No, there isn't any flag that is only used by mpicc and friends, but not used to compile MPICH2 itself. You might be able to do this by poking around with MPI_OTHERLIBS or LIBS, but that's not tested. I think what you suggested is a good idea. Unless someone else on the MPICH2 team has any better ideas, I can try to squeeze this into our next release (1.1.1) that should be out in a few weeks.

[Kraus Philipp]

Hi,

Am 27.06.2009 um 02:09 schrieb mpich2:

> No, there isn't any flag that is only used by mpicc and friends, but
> not
> used to compile MPICH2 itself. You might be able to do this by poking
> around with MPI_OTHERLIBS or LIBS, but that's not tested. I think
> what you
> suggested is a good idea. Unless someone else on the MPICH2 team has
> any
> better ideas, I can try to squeeze this into our next release
> (1.1.1) that
> should be out in a few weeks.

That sounds good, I must develope my codes for both (OpenMPI and
MPICH2).
I think the best solution would be to set in a config file a flag for
reading the OMPI_
flags (x)or set special ones for MPICH2

Thanks

Phil

[chan]

If I understand OMPI_LDFLAGS correctly, it is used to override the default LDFLAGS (set by configure) used in the OpenMPI's compiler wrappers. Since you are using mpicxx or mpic++ to link Boost with your app, you shouldn't need to override OMPI_LDFLAGS. You can simply do what said,

mpic++ -o <target> <source> -lboost_mpi-xgcc40-mt-1_39 .....

for both OpenMPI and MPICH2. I don't understand your motivation in setting OMPI_LDFLAGS!

There is way to insert your LDFLAGS to MPICH2's compiler wrappers, but I don't think it is necessary in your case though.

A.Chan

[Kraus Philipp]

Am 27.06.2009 um 13:23 schrieb mpich2:
> If I understand OMPI_LDFLAGS correctly, it is used to override the
> default
> LDFLAGS
> (set by configure) used in the OpenMPI's compiler wrappers.  Since
> you are
> using mpicxx or mpic++ to link Boost with your app, you shouldn't
> need to
> override OMPI_LDFLAGS.  You can simply do what said,
>
> mpic++ -o <target> <source> -lboost_mpi-xgcc40-mt-1_39 .....
>

Yes, but on our cluster we have a lot of users and would set the libs
in the general profile file.
I can't set the LDFLAGS because on this situation every ld run links
the libs into the file and
we have some projects that don't need the boost libs.


> for both OpenMPI and MPICH2.  I don't understand your motivation in
> setting OMPI_LDFLAGS!

We developing with Open MPI and MPICH, so I would like only on this
projects with the Boost
so I must add on every mpic++ run (for Open MPI and MPI CH) the -l
options or on Open MPI
I can set the OMPI_LDFLAGS in my profile that is used by the mpic++
The mpic++ of MPI CH don't read the OMPI_LDFLAGS.

The goal is that wie can set into the /etc/profile the LDFLAGS and
seperated the flags for mpic++
In our case the LDFLAGS for Open MPI and MPI CH must be the same.

for example:

export LD_FLAGS=-lmylib123
export OMPI_LDFLAGS=-lboost
export MPICH_LDFLAGS=$OMPIFLAGS

the students should be run only mpic++ in this case:
/path-to-mpich-or-openmpi/mpic++ -o <target> <source> without touching
the MPI configuration

We need at this time both (OpenMPI and MPICH)

Thanks

Phil

[anonymous]

>  We developing with Open MPI and MPICH, so I would like only on this
>  projects with the Boost
>  so I must add on every mpic++ run (for Open MPI and MPI CH) the -l
>  options or on Open MPI
>  I can set the OMPI_LDFLAGS in my profile that is used by the mpic++
>  The mpic++ of MPI CH don't read the OMPI_LDFLAGS.
>
>  The goal is that wie can set into the /etc/profile the LDFLAGS and
>  seperated the flags for mpic++
>  In our case the LDFLAGS for Open MPI and MPI CH must be the same.
>
>  for example:
>
>  export LD_FLAGS=-lmylib123
>  export OMPI_LDFLAGS=-lboost
>  export MPICH_LDFLAGS=$OMPIFLAGS
>
>  the students should be run only mpic++ in this case:
>  /path-to-mpich-or-openmpi/mpic++ -o <target> <source> without
> touching the MPI configuration

Are you using Makefile ?  You can give your students a default Makefile
that have this line.

LDFLAGS = -lmylib123 -lboosts

<target>: <source>
          mpic++ -o $@ $? $LDFLAGS.

Then the student only needs to type "make".  I don't understand why students
need to touch MPI configuration.

Again if I understand OMPI_LDFLAGS correctly, it is meant for mixing language
programming that the default LDFLAGS setting isn't working.  Or OpenMPI was
compiled
with one compiler but is used with a "similar" compiler with different runtime
environment that requires a different LDFLAGS.  Setting OMPI_LDFLAGS in
/etc/profile
for Boost just sounds like a BAD idea.

If you insist on student not using Makefile but simply using mpic++ without
explicit
LDFLAGS, you should look into using the -profile option in mpicxx.  See the
mpicxx
script for details.  You still need to do "mpicxx -profile=boost -o <target>
<source>"
You need to create a profile for boost...  I think this should work.

A.Chan

[Anthony Chan]

>  If you insist on student not using Makefile but simply using mpic++
>  without
>  explicit
>  LDFLAGS, you should look into using the -profile option in mpicxx.
> See
>  the
>  mpicxx
>  script for details.  You still need to do "mpicxx -profile=boost -o
>  <target>
>  <source>"
>  You need to create a profile for boost...  I think this should work.

Forgot there is a even simpler way.  OpenMPI's compiler wrappers are binarys
so it has to provide all these OMPI_*FLAGS so user can still use the compiler
wrappers.  MPICH2's compiler wrappers are scripts.  You could simply copy
MPICH2 mpicxx and create your boost_mpicxx and add "-lmylib123 -lboost"
in final link command in your boost_mpicxx and have your student uses
boost_mpicxx to compile (instead of MPICH2's mpicxx).

A.Chan

[Kraus Philipp]

Am 27.06.2009 um 22:57 schrieb mpich2:

> --------------------------------------------------------
> +-------------------
> Reporter:  Kraus Philipp <philipp.kraus@flashpixx.de>  |
> Owner:  balaji
>     Type:  bug                                         |
> Status:  new
> Priority:  major                                       |
> Milestone:  mpich2-1.1.1
> Component:  mpich2                                      |
> Resolution:
> Keywords:                                              |
> --------------------------------------------------------
> +-------------------
>
>
> Comment (by ):
>
> {{{
>
>
>> We developing with Open MPI and MPICH, so I would like only on this
>> projects with the Boost
>> so I must add on every mpic++ run (for Open MPI and MPI CH) the -l
>> options or on Open MPI
>> I can set the OMPI_LDFLAGS in my profile that is used by the mpic++
>> The mpic++ of MPI CH don't read the OMPI_LDFLAGS.
>>
>> The goal is that wie can set into the /etc/profile the LDFLAGS and
>> seperated the flags for mpic++
>> In our case the LDFLAGS for Open MPI and MPI CH must be the same.
>>
>> for example:
>>
>> export LD_FLAGS=-lmylib123
>> export OMPI_LDFLAGS=-lboost
>> export MPICH_LDFLAGS=$OMPIFLAGS
>>
>> the students should be run only mpic++ in this case:
>> /path-to-mpich-or-openmpi/mpic++ -o <target> <source> without
>> touching the MPI configuration
>
> Are you using Makefile ?  You can give your students a default
> Makefile
> that have this line.
>
> LDFLAGS = -lmylib123 -lboosts
>
> <target>: <source>
>           mpic++ -o $@ $? $LDFLAGS.
>
> Then the student only needs to type "make".  I don't understand why
> students
> need to touch MPI configuration.
>
> Again if I understand OMPI_LDFLAGS correctly, it is meant for mixing
> language
> programming that the default LDFLAGS setting isn't working.  Or
> OpenMPI
> was
> compiled
> with one compiler but is used with a "similar" compiler with different
> runtime
> environment that requires a different LDFLAGS.  Setting OMPI_LDFLAGS
> in
> /etc/profile
> for Boost just sounds like a BAD idea.
>
> If you insist on student not using Makefile but simply using mpic++
> without
> explicit
> LDFLAGS, you should look into using the -profile option in mpicxx.
> See
> the
> mpicxx
> script for details.  You still need to do "mpicxx -profile=boost -o
> <target>
> <source>"
> You need to create a profile for boost...  I think this should work.

Okay I don't know about the profile option. at the moment I create a
script that
reads the OMPI flags and do the mpicxx call with this flags

Thanks for helping, I will solve the problem with the profile option


Phil

[balaji]

After some discussion here, I've finally added flags that only get used within mpicc and friends, and are not for the MPICH2 library's compilation itself. This has been committed as of r4856. The flags are called MPICH2_MPICC_FLAGS, etc. This should appear in mpich2-1.1.1. Resolving this ticket.

-- Pavan