a trouble with 64-bit system, Alex Mazheika

[Alexej Mazheika <alex.mazheika@…>]

Hi

My name is Alex Mazheika. I use mpich2 for quantum chemical calculations
under Linux (I compiled the program package Espresso (
www.quantum-espresso.org) with mpich2 and apply it for parallel processes
calculations). I noticed a strange phenomenon. The CPU of my PC is Intel
Core Quad (64-bit). When I make my calculations under 64-bit linux-system
(openSUSE 11.0, Fedora 10, Mandriva) it lasts longer than under 32-bit one
(openSUSE-11.0). In both cases I compiled mpich2-1.1 (or earlier versions)
from source-files with gcc, gfortran. The specifying
> mpiexec -arch x86_64 ...
doesn't change anything. Is it OK? Or should I change any options during
compiling of mpich2 or of my Espresso-package? Or, maby, it needs to specify
anything in command console by starting of mpiexec?

thank you in advance

regards,
Alex Mazheika, Research institut for Physical Chemical Problems of BSU,
Leningradskayastr. 14, 220050 Minsk, Belarus

[Alexej Mazheika]

Added by email2trac

[Alexej Mazheika]

This message has 1 attachment(s)

[Rajeev Thakur]

How much is the time difference between the 64-bit and 32-bit versions?
You may also want to try with a single 64-bit OS and compiling
separately for 32-bit or 64-bit via the -m32 and -m64 options to gcc and
gfortran.

[Alexej Mazheika]

Added by email2trac

[Alexej Mazheika]

>
>   How much is the time difference between the 64-bit and 32-bit versions?
>   You may also want to try with a single 64-bit OS and compiling
>   separately for 32-bit or 64-bit via the -m32 and -m64 options to gcc and
>   gfortran.



The time  difference between the 64-bit and 32-bit versions is not so much
but enough: it does an identical procedure under 64-bit OS during 8.4-8.5
min and under 32-bit OS during 7.5-7.6 min.
Sorry, but I didn't understand about compiling. I compile like it
is described in the install.pdf, through commands:
>/home/you/libraries/mpich2-1.1/configure -prefix=/home/you/mpich2-install
2>&1 | tee c.txt
and then >make 2>&1 | tee m.txt and >make install 2>&1 | tee mi.txt
How should I compile with -m32 and -m64 options? These optitions are invalid
for commands configure and make. Or should I compile separately each
source-file of mpich2 with gcc and gfortran? I don't completely understand
how to carry out such compilation, and whether it will work after this
separate compilation. Is there any description of it in ../mpich2-1.1/doc/ ?

Alex

[Rajeev Thakur]

You can use a 64-bit OS and set the environment variables CC, CXX, F77,
F90 to "gcc -m32", "g++ -m32", "gfortran -m32", "gfortran -m32". Then
run configure, make clean, make, make install as usual. That will create
32-bit binaries on a 64-bit OS. See how that performs compared with
regular 64-bit binaries on the 64-bit OS.

Rajeev

[thakur]

Resolving until we hear further

[Alexej Mazheika]

Added by email2trac

[Alexej Mazheika]

>
> You can use a 64-bit OS and set the environment variables CC, CXX, F77,
>   F90 to "gcc -m32", "g++ -m32", "gfortran -m32", "gfortran -m32". Then
>   run configure, make clean, make, make install as usual. That will create
>   32-bit binaries on a 64-bit OS. See how that performs compared with
>   regular 64-bit binaries on the 64-bit OS.
>
>   Rajeev
>

I used 64-bit OS and compiled mpich2 like you had described, before
installation I set:
under bash >set gcc -m64 (or g++,gfortran)
under tcsh >setenv gcc -m64
With -m32 I have done analogously. And no change was considered. Under
32-bit system my processes last quicklier.
If I set ">set CC -m64" (CXX, F77, F90), it writes by configuration:
"checking for gcc... -m64
checking for C compiler default output file name...
configure: error: C compiler cannot create executables
See `config.log' for more details."
So it don't want to run qiucklier under 64-bit than under 32-bit OS by any
setting of environment variables. It seems me that maybe the reason lies in
my Espresso-program (www.pwscf.org), because I didn't try to run any other
multiple process through mpich2, only Espresso. Or is there any other ideas?

Alex

[anonymous]

----- "mpich2" <mpich2-maint@mcs.anl.gov> wrote:


> > You can use a 64-bit OS and set the environment variables CC, CXX, F77,
> > F90 to "gcc -m32", "g++ -m32", "gfortran -m32", "gfortran -m32". Then
> > run configure, make clean, make, make install as usual. That will create
> > 32-bit binaries on a 64-bit OS. See how that performs compared with
> > regular 64-bit binaries on the 64-bit OS.
> >
> >   Rajeev
>
>  under bash >set gcc -m64 (or g++,gfortran)
>  under tcsh >setenv gcc -m64
>  With -m32 I have done analogously. And no change was considered.
> Under
>  32-bit system my processes last quicklier.
>  If I set ">set CC -m64" (CXX, F77, F90), it writes by configuration:
>  "checking for gcc... -m64
>  checking for C compiler default output file name...
>  configure: error: C compiler cannot create executables
>  See `config.log' for more details."

You did all wrong.  You forgot your CC/CXX/F77/F90 variable in
your command.  Do

bash: export CC="gcc -m32"
tcsh: setenv CC "gcc -m32"

for all CC/CXX/F77/F90 before configure.  Or you can set these
variables as part of your configure command,
e.g. configure CC="gcc -m32" CXX="g++ -m32" ..... --prefix=...
Make sure your 64bit OS has 32-bit libraries installed, check
with your system admin.

A.Chan

[Alexej Mazheika]

Added by email2trac

[Alexej Mazheika]

>
> You did all wrong.  You forgot your CC/CXX/F77/F90 variable in
>   your command.  Do
>
>   bash: export CC="gcc -m32"
>   tcsh: setenv CC "gcc -m32"
>
>   for all CC/CXX/F77/F90 before configure.  Or you can set these
>   variables as part of your configure command,
>   e.g. configure CC="gcc -m32" CXX="g++ -m32" ..... --prefix=...
>   Make sure your 64bit OS has 32-bit libraries installed, check
>   with your system admin.
>
>   A.Chan
>

Thank you for your detailed letter. I did everything like you had
recommended. If I set under 64-bit OS >export CC="gcc -m64" (the same for
CXX/F77/F90) it configured without any problems. But if I set >export
CC="gcc -m32" it wrote by configuration:
"checking for gcc... -m32
checking for C compiler default output file name...
configure: error: C compiler cannot create executables
See `config.log' for more details."
So it didn't create 32-bit binaries on a 64-bit OS.
Under 32-bit OS it was quite the contrary: if I set -m32 it was OK; if I set
-m64 - configure error.
By such compilation the velosity of my multiprocess calculations with mpich2
was as usual quicklier under 32-bit OS than under 64-bit one. The same
process needs 11 min under 32-bit and 12 min under 64-bit OS. So, I think,
the reason doesn't lie in the procedure of configuration before installation
of mpich2. Without set of -m32 or -m64 the results of installation of mpich2
were the same.

Alex Mazheika

[Anthony Chan]

----- "mpich2" <mpich2-maint@mcs.anl.gov> wrote:

>  But if I set export
>  CC="gcc -m32" it wrote by configuration:
>  "checking for gcc... -m32
>  checking for C compiler default output file name...
>  configure: error: C compiler cannot create executables
>  See `config.log' for more details."

configure still think that CC is set to "-m32" not "gcc -m32".

A.Chan

[Alexej Mazheika]

>
> configure still think that CC is set to "-m32" not "gcc -m32".
>
>   A.Chan
>

Now I tried to compile mpich2 under 64-bit system with commands
>configure gcc="gcc -m32" cxx="g++ -m32" ..... --prefix=...
or  >configure cc="gcc -m32" cxx="g++ -m32" ..... --prefix=...
It was everything OK by installation of mpich2. Then by configuration of my
programm no errors were found, but during installation with mpich2 it has
written:
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld:
skipping incompatible /home/user/mpich2-install/lib/libmpichf90.a when
searching for -lmpichf90
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld:
skipping incompatible /home/user/mpich2-install/lib/libmpichf90.a when
searching for -lmpichf90
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld:
cannot find -lmpichf90
exit status
So installation was broken. Can you write me how to do the
mpich2-installation under 64-bit OS with -m32 exactly, point-by-point? Or
maybe I have done everything OK and there is another problem?

A.Mazheika

[Alexej Mazheika]

Added by email2trac

[goodell]

Hi Alexej,

I suspect that your remaining problem with F90 was fixed in r4896 (missing eval in configure.in). Would you be willing to try two more things?

1. Run file /home/user/mpich2-install/lib/* and send us the output. This should hopefully confirm my theory that you are getting a 64-bit f90 library despite trying to build it as 32-bit. Please also send your config.log.
2. Try building the following tarball instead of mpich2-1.1. It should contain the r4896 fix, which will hopefully solve the remainder of your problem: http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/nightly/trunk/mpich2-trunk-r4938.tar.gz

-Dave

[Alexej Mazheika]

>
>   I suspect that your remaining problem with F90 was fixed in r4896
> (missing
>   eval in configure.in).  Would you be willing to try two more things?
>
>   1. Run {{{file /home/user/mpich2-install/lib/*}}} and send us the output.
>   This should hopefully confirm my theory that you are getting a 64-bit f90
>   library despite trying to build it as 32-bit.  Please also send your
>   config.log.
>   2. Try building the following tarball instead of mpich2-1.1.  It should
>   contain the r4896 fix, which will hopefully solve the remainder of your
>   problem:
>
> http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/nightly/tru
nk/mpich2-trunk-r4938.tar.gz
>
>   -Dave
>

I'm sorry, I misinformed you a little in my previous letter. I retried
to install mpich2 under 64-bit OS with commands -m32 and -m64
(CC/CXX/F77/F90 I wrote with small letters):
>...configure cc="gcc -m32" cxx="g++ -m32" f77="gfortran -m32" f90="gfortran
-m32" gfortran="gfortran -m32" -prefix=...
and
>...configure cc="gcc -m64" cxx="g++ -m64" f77="gfortran -m64" f90="gfortran
-m64" gfortran="gfortran -m64" -prefix=...
And then I compiled my program in each case. It was everything without
problems (in my previous letter it was my mistake by installation of
mpich2). After that I checked the time during a multiprocess calculation in
both cases. So under 64-bit OS both by installation with -m32 and with -m64
the same calculation lasts ~9 min. Under 32-bit OS it lasts ~8 min.
The multiprocess calculation with mpich2-trunk-r4938 I test today under
64-OS and tomorrow I report results.

Alexej Mazheika

[Alexej Mazheika]

Added by email2trac

[Alexej Mazheika]

Added by email2trac

[Alexej Mazheika]

>
>   2. Try building the following tarball instead of mpich2-1.1.  It should
>>   contain the r4896 fix, which will hopefully solve the remainder of your
>>   problem:
>>
>> http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/nightly/tr
unk/mpich2-trunk-r4938.tar.gz
>>
>>   -Dave
>>
>
I have installed mpich2-trunk-r4938 in my 64-bit OS without any set (by
default it must be -m64) and then I've compiled my programm. Like in the
case with mpich2-1.1 my calculations last longer than under 32-bit OS (9 min
vs 8 min), no changes at time were noticed.

Al. Mazheika

[goodell]

I'm not sure why there would be any significant performance discrepancy in this case, especially in favor of 32-bit. If anything, I would expect the 64-bit code to run a bit faster in most cases.

The best I can recommend to you is to profile your application under both operating systems and try to determine what part of your application is running slower. You can use tools like gprof and MPE to try to dig in to this performance problem.