root/mpich2/branches/dev/kumudb/doc/logging/logging.tex @ 4870

\documentclass[dvipdfm,11pt]{article}
\usepackage[dvipdfm]{hyperref} % Upgraded url package
\parskip=.1in

\begin{document}
\markright{MPICH2 Logging}
\title{MPICH2 Logging\\
Version 0.1\\
DRAFT of \today\\
Mathematics and Computer Science Division\\
Argonne National Laboratory}

\author{David Ashton}


\maketitle

\cleardoublepage

\pagenumbering{arabic}
\pagestyle{headings}


\section{Introduction}
\label{sec:introduction}

This manual assumes that MPICH2 has already been installed.  For
instructions on how to install MPICH2, see the MPICH2 Installer's Guide,
or the README in the top-level MPICH2 directory.  This manual will
explain how the internal logging macros are generated and how the user
can generate log files viewable in Jumpshot.  Use of Jumpshot is 
described in the mpe documentation.


\section{Configuring mpich2 to create log files}
\label{sec:configuring}

When users run configure they can specify logging options.  There are three 
configure options to control logging.

\begin{description}
\item[\texttt{--enable-timing=<timing\_type>}]\mbox{}\\
Add this option to enable timing.  The two options for timing\_type are 
\texttt{log} and \texttt{log\_detailed}.  The \texttt{log} option will log 
only the MPI functions just like the MPE logging interface does.  The 
\texttt{log\_detailed} will log every function in mpich2.  This option gives 
fine grained logging information and also creates large log files.  It must 
be used in conjunction with a timer-type that can log very short intervals 
on the order of 100's of nanoseconds.

\item[\texttt{--with-logging=<logger>}]\mbox{}\\
Specify the logging library to use.  Currently the only logger option is \texttt{rlog}.

\item[\texttt{--enable-timer-type=<timer\_type>}]\mbox{}\\
Specify the timer type.  The options are
\begin{itemize}
\item \texttt{gethrtime} -
Solaris timer (Solaris systems only)
\item \texttt{clock\_gettime} -
Posix timer (where available)
\item \texttt{gettimeofday} -
Most Unix systems
\item \texttt{linux86\_cycle} -
Linux x86 cycle counter*
\item \texttt{linuxalpha\_cycle} -
Like linux86\_cycle, but for Linux Alpha*
\item \texttt{gcc\_ia64\_cycle} -
IA64 cycle counter*
\end{itemize}
* Note that CPU cycle counters count cycles, not elapsed time.
Because processor frequencies are variable, especially with modern
power-aware hardware, these are not always reliable for timing and so
should only be used if you're sure you know what you're doing.

\end{description}

Here is an example:
\begin{verbatim}
mpich2/configure
    --enable-timing=log
    --with-logging=rlog
    --enable-timer-type=gettimeofday
    ...
\end{verbatim}

\section{Generating log files}
\label{sec:genlogs}
Run your mpi application to create intermediate \texttt{.irlog} files.

\begin{verbatim}
mpicc myapp.c -o myapp
mpiexec -n 3 myapp
\end{verbatim}
There will be .irlog files created for each process:
\begin{verbatim}
log0.irlog
log1.irlog
log2.irlog
\end{verbatim}

\section{RLOG tools}
\label{sec:tools}
For performance reasons each process produces a local intermediate log file 
that needs to be merged into a single rlog file.  Use the rlog tools to merge 
the \texttt{.irlog} files into an \texttt{.rlog} file.  The rlog tools are 
found in \texttt{mpich2\_build/src/util/logging/rlog}. Currently they are not 
copied to the install directory.

\begin{description}
\item[\texttt{irlog2rlog}]\mbox{}\\
This tool combines the intermediate \texttt{.irlog} files into a single 
\texttt{.rlog} file. The usage is: ``\texttt{irlog2rlog outname.rlog 
input0.irlog input1.irlog ...}'' A shortcut is provided: ``\texttt{irlog2rlog 
outname.rlog <num\_files>}''.  Execute \texttt{irlog2rlog} without any 
parameters to see the usage options.

\item[\texttt{printrlog}]\mbox{}\\
This tool prints the contents of an \texttt{.rlog} file.

\item[\texttt{printirlog}]\mbox{}\\
This tool prints the contents of an \texttt{.irlog} file.
\end{description}

Continuing the example from the previous section:
\begin{verbatim}
irlog2rlog myapp.rlog 3
\end{verbatim}
will convert \texttt{log0.irlog}, \texttt{log1.irlog} and \texttt{log2.irlog} 
to \texttt{myapp.rlog}.

\section{Viewing log files}
This section describes how to view a log file

\texttt{.rlog} files can be printed from a command shell using the 
\texttt{printrlog} tool but the more interesting way to view the log files 
is from Jumpshot.  Jumpshot displays slog2 files and has a built in converter 
to convert \texttt{.rlog} files to \texttt{.slog2} files.  Start Jumpshot and 
open your \texttt{.rlog} file.  Jumpshot will ask you if you want to convert
the file and you say yes.

\section{Logging state code generation}
\label{sec:genstates}

This section can be skipped by users.  It describes the internal scripts used
to develop the logging macros.

This is how the \texttt{maint/genstates} script works:

\begin{enumerate}
\item \texttt{maint/updatefiles} creates \texttt{genstates} from 
\texttt{genstates.in} replacing \texttt{@PERL@} with the appropriate path to
perl and then runs \texttt{genstates}.

\item \texttt{genstates} finds all \texttt{.i}, \texttt{.h} and \texttt{.c} files 
in the mpich2 directory tree, searches for \texttt{\_STATE\_DECL} in each 
file and builds a list of all the MPID\_STATEs.  It validates that the states
start in a \texttt{\_STATE\_DECL} statement, followed by a \texttt{FUNC\_ENTER}
statement, and then at least one \texttt{FUNC\_EXIT} statement.  Errors are printed
out if the code does not follow this format except for macros.  State declarations
in macros are assumed to be correct.

\item \texttt{genstates} finds all the \texttt{describe\_states.txt} 
files anywhere in the mpich2 tree.  \texttt{describe\_states.txt} files are 
optional and are used to set the output name of the state and its associated 
color.

\item The \texttt{describe\_states.txt} file format is this:
\begin{verbatim}
MPID_STATE_XXX <user string for the state> <optional rgb color>
\end{verbatim}
 Here is an example line:
\begin{verbatim}
 MPID_STATE_MPI_SEND MPI_Send 0 0 255
\end{verbatim}
If you don't specify a state in a \texttt{describe\_states.txt} file then
the state user name will be automatically created by stripping off the 
\texttt{MPID\_STATE\_} prefix and the color will be assigned a random value.

\item \texttt{genstates} ouputs \texttt{mpich2/src/include/mpiallstates.h} 
with this \texttt{enum} in it:
\begin{verbatim}
enum MPID_TIMER_STATE
{
    MPID_STATE_XXX,
    ...
};
\end{verbatim}

\item \texttt{genstates} outputs 
\texttt{mpich2/src/util/logging/describe\_states.c} with the
\texttt{MPIR\_Describe\_timer\_states()} function in it.  Currently, only 
the rlog version of \texttt{MPIR\_Describe\_timer\_states()} is generated.

\end{enumerate}

\end{document}