wiki:BleuJay
Last modified 10 years ago Last modified on 05/05/09 17:22:39

TracNav(TracNavPerformance)?

"BleuJay: Automatic Generation of Performance and Memory Models"

Overview

BleuJay software provides support for automatic generation of performance and memory models of parallel scientific applications. The compilation of software includes command line utilities and performance measurement libraries that currently supports performance measurement for OpenMP and MPI codes. The performance data generated by BleuJay includes OpenMP metrics and high-level performance and memory metrics based on performance hardware counters with a full mapping back to source code. The design goal underlying BleuJay is to significantly simplify the performance tuning cycle in the optimization of parallel scientific applications.

Download

(Open-source license)

Development version (unstable) can be checked out anonymously with:

svn co https://svn.mcs.anl.gov/repos/performance/BleuJay/trunk BleuJay-trunk

Contact Van Bui if you wish to contribute to the BleuJay Project.

Software Requirements

BleuJay has been tested on Linux/x86-64 based systems. Before you start, you should have already installed the following software packages and run their test suites to make sure everything is working properly:

  1. Perfsuite. Properly install the libperfsuite and libpshwpc libraries from the Perfsuite software package. Configure Perfsuite with support for PAPI and MPI.

http://perfsuite.ncsa.uiuc.edu/

In addition, libpsx is also required. Libpsx is an extension to the core Perfsuite libraries that can be used to expose additional contextual information in addition to the raw performance data. Libpsx is developed and supported by the original developers of Perfsuite. An alpha/prototype version of the library can be downloaded at this temporary location:

http://www.mcs.anl.gov/~vbui/libpsx-0.2.tar.gz

Configure libpsx with Perfsuite and Libunwind support.

  1. OpenMP runtime library supporting the OpenMP/Collector Profiling Interface. To date, the OpenUH compiler and OpenMP compiler from Sun Microsystems provide support for the OpenMP/Collector profiling interface. OpenUH can be downloaded from:

svn co http://svn.open64.net/svnroot/open64/branches/OpenUH/

http://www2.cs.uh.edu/~openuh/

Configure to enable support for shared libraries.

  1. Libunwind. This library supports retrieval of the runtime callstack. Libunwind is also prerequisite software for libpsx.

http://www.nongnu.org/libunwind

  1. PAPI (Performance API), from the Innovative Computing Laboratory at the University of Tennessee/Knoxville?, version 3.

http://icl.cs.utk.edu/papi/

  1. CUBE, performance visualization software.

http://www.fz-juelich.de/jsc/scalasca/software/download/

  1. Expat (XML parsing library originally written by James Clark, now continued development by a group of others). This is also pre-requisite software for building Perfsuite.

http://www.libexpat.org/

  1. Binutils and libiberty. These are usually included with Linux distributions but there may not be recent versions available; you may want to update. This is also pre-requisite software for building Perfsuite.

http://www.gnu.org/software/binutils

  1. Python version 2.5 or higher.
  1. MPI. Configure to enable support for shared libraries. BleuJay currently supports performance measurement of hybrid MPI/OpenMP codes. Plans are currently underway for generating support for MPI specific metrics.

Build and Installation

Building and installing the libraries

Note: $BLEUJAYROOT is the location where BleuJay source files are located on your system.

  1. Go into the $BLEUJAYROOT/src directory.
  1. In the Makefile, set the following system dependent variables:
       $BLEUJAYDIR  - BleuJay installation directory
       $BLEUJAYROOT - location of BleuJay source file downloads
       $PSDIR - PerfSuite installation directory
       $PSXDIR - libpsx installation directory
       $PAPIDIR - PAPI installation directory
       $UNWDIR - Libunwind installation directory
       $BFDDIR - Binutils installation directory
       $OMPDIR - OpenMP installation directory
       $EXPATDIR - EXPAT installation directory
    
  1. To build the library supporting measurement of OpenMP codes:
       a. set in the Makefile $HYBRIDFLAG = -DHYBRID=0
       b. % make chwpc_r
    
  2. To build the library supporting measurement of MPI/OpenMP codes:
       a. set in the Makefile $HYBRIDFLAG = -DHYBRID=1
       b. % make chwpc_h
    
  1. % make install
  1. Optional: % make clean

Building and installing the command line utilities

  1. Go into the $BLEUJAYROOT/tools directory.
  1. In the Makefile, set the following variables:
       $BLEUJAYDIR  - BleuJay installation directory
       $CUBEDIR - CUBE installation directory
    
  2. % make
  1. % make install
  1. Optional: % make clean

Instructions for using BleuJay

NOTE: $BLEUJAYDIR is the location of the BleuJay installation.

  1. Add $BLEUJAYDIR/bin to $PATH
  1. Add $BLEUJAYDIR/lib to $LD_LIBRARY_PATH
  1. Set $BLEUJAYHOME to $BLEUJAYDIR
  1. To collect data for OpenMP codes:
       % bjrun ./foo
    

A single performance log file should generate with the following naming convention: program_name.pid.0

  1. To collect data for MPI/OpenMP codes
      % bjrun -h mpirun -np #procs ./foo
    

Performance log files should generate for each process with the following naming convention: program_name.pid.mpi_rank_id

  1. To convert the performance log files generated after running your program to cube format:
      % convert2cube file0 file1
    

A single CUBE file should generate with the following naming convention: program_name.cube

  1. Performance data can be visualized using CUBE, one of the prequisite software for BleuJay.
      % cube foo.cube
    

Papers

  • K. Huck, O. Hernandez, V. Bui, S. Chandrasekaran, B. Chapman, A. D. Malony, L.C. Mcinnes, and B. Norris. Capturing Performance Knowledge for Automated Analysis, Supercomputing, 2008.
  • V. Bui, O. Hernandez, K. Huck, L. Li, B. Norris, L.C. Mcinnes, and B. Chapman. A Component Infrastructure for Performance and Power Modeling of Parallel Scientific Applications, Component Based High Performance Computing Conference, 2008.

Presentations

  • V. Bui. Automatic Generation of Performance/Memory? Models for Parallel Scientific Applications. SIAM Conference on Computational Science and Engineering, 2008.
  • B. Norris. A Component Infrastructure for Performance and Power Modeling of Parallel Scientific Applications. Component Based High Performance Computing Conference, 2008.

Attachments