ARL DSRC - Special Projects

Computational Science Environment (CSE)

CSE Logo In efforts to provide a common platform for utilizing and developing data analysis and assessment applications in heterogeneous High Performance Computing (HPC) environments, the Classified Data Analysis and Assessment Center (CDAAC) has developed the Computational Science Environment (CSE). CSE is a new module based software release structure for the software platform once known as the Interdisciplinary Computing Environment (ICE).

CSE provides a stable suite of data analysis and assessment tools, applications and libraries that are common across most HPC environments and developers workstations. In addition, CSE provides an experimental, cutting-edge suite of developer tools that can enhance the data analysis and assessment process. CSE includes the eXtensible Data Model and Format (XDMF), a common data hub where HPC codes and tools can efficiently exchange data values and meaning. CSE consist of many tools like Python, VTK and ParaView that can assist with day-to-day visualization requirement. CSE provides a common platform that enables the development of full featured portable HPC applications.

CSE is available on the following DSRC systems:

CSE Availability
System	Install Location
Utility Servers (ALL)	/app/CSE/CSE
Diamond	/usr/local/usp/cseb_cse/CSE
Garnet	/usr/local/usp/cseb_cse/CSE
Haise	/site/cse
Harold	/usr/cta/CSE
Kilrain	/site/cse
MRAP	/usr/cta/CSE
Pershing	/usr/cta/CSE
Raptor	/app2/cse/CSE
TOW	/usr/cta/CSE

CSE can is also supported on RHEL5 64-bit systems.

1. Initial CSE module Setup

1.1. Initial module setup at ARL (harold, tow)

module load cseinit
module load cse-tools

1.2. Initial module setup at all other DSRCs

module use INSTALLATION LOCATION/modules/COTS
module load cseinit cse-tools

Note: The cseinit module assists the user by loading the compiler and mpi version that were used to build CSE. If the user does not wish to have CSE assist with the compiler and mpi versions a module named cseinit-devel is available. It should be noted when using the cseinit-devel that the user could possibly encounter compiler and mpi conflicts, undefined references, or other issues.

1.2.1. Initialize modules using .cshrc

To initialize modules on your machine add the following to your .cshrc :

module use INSTALLATION LOCATION/modules/COTS

1.2.2. Initialize modules from the command line

Initialize modules from the command line by executing the following command:

module use INSTALLATION LOCATION/modules/COTS

2. Machines without modules (64-bit Linux Workstations)

On non-HPC systems such as desktop workstations, the modules software is included with the CSE distribution. The CSE distribution is available using rdist for RHEL5 linux workstations from an ARL server. For information on how to gain access to the CSE software, please contact the vis@arl.army.mil. Once CSE is installed, the user is required to initialize modules using one of the following methods:

2.1. Initialize modules using .cshrc

To initialize modules on your machine add the following to your .cshrc :

if (-e /usr/cta/CSE/modules/init/tcsh) then
source /usr/cta/CSE/modules/init/tcsh
module use /usr/cta/CSE/modules/COTS
endif

2.2. Initialize modules from the command line

/usr/cta/CSE/modules/init/tcsh
module use /usr/cta/CSE/modules/COTS

3. Loading and Viewing CSE modules

The CSE module layout is based on simplicity (see packages only if you want to see them) and uses a consistent naming schema that attempts to avoid future naming/package conflicts. The central idea is to simplify the process associated with establishing the proper environment to execute a particular application. The end-user will not be required to establish environment variables or modify library paths to get common applications to work properly. To illustrate how to load different CSE packages several examples are provided.

3.1. Example 1: View and load a CSE application module

To view and load a cse application module:

Load main cse module:

module load cseinit

List the modules:

module avail
  
--------------------------- /usr/cta/CSE/Analysis/modules/rel-tools ------------------------------
cse-analysis/gnuplot/4.4.0        cse-analysis/matplotlib/latest   cse-analysis/scipy/0.7.0
cse-analysis/gnuplot/latest       cse-analysis/octave/3.2.4        cse-analysis/scipy/latest
cse-analysis/matplotlib/0.99.1.2  cse-analysis/octave/latest

----------------------------- /usr/cta/CSE/Apps/modules/rel-apps ---------------------------------
cse/paraview/3.6.2     cse/paraview/latest        cse/paraview_mesa/3.8.0
cse/paraview/3.8.0     cse/paraview_mesa/3.6.2    cse/paraview_mesa/latest

------------------------------- /usr/cta/CSE/modules/rel-apps ------------------------------------
cse/doxygen/1.5.9    cse/openmpi/1.4.1   cse/paraview/latest       cse/xframework/2008.10  cse-tools
cse/doxygen/latest   cse/openmpi/latest  cse/paraview_mesa/3.6.2   cse/xframework/latest
cse/graphviz/2.22.2  cse/paraview/3.6.2  cse/paraview_mesa/3.8.0   cse-beta
cse/graphviz/latest  cse/paraview/3.8.0  cse/paraview_mesa/latest  cse-info

--------------------------------- /usr/cta/CSE/modules/COTS --------------------------------------
cseinit       cseinit-devel

Load desired module:

Loading the main CSE module (cseinit) will make the application list (ParaView, Visit, OpenMPI ...) available to load. CSE will commonly provide several versions of the same package, by default the latest version of a particular package is cse/package/latest. Loading cse-tools is discussed in the following example:

3.2. Example 2: View and load a CSE tool module

To view and load a cse tool module:

Load main cse module:

module load cseinit

Load cse tools module:

module load cse-tools

List the modules:

module avail
  
-------------------------- /usr/cta/CSE/modules/rel-tools ---------------------------------
cse/cmake/2.8.2      cse/mpi4py/latest      cse/qt/4.6.3         cse/tcltk/8.4.19
cse/cmake/latest     cse/netdmf/2009.12.16  cse/qt/latest        cse/tcltk/latest
cse/freealut/1.1.0   cse/netdmf/latest      cse/scons/1.0.0      cse/valgrind/3.5.0
cse/freealut/latest  cse/numpy/1.3.0        cse/scons/latest     cse/valgrind/latest
cse/git/1.6.5.2      cse/numpy/latest       cse/sip/4.7.4        cse/vtk/5.4.2
cse/git/latest       cse/openal/0.0.8       cse/sip/4.9.1        cse/vtk/5.6.0
cse/hdf5/1.6.8       cse/openal/latest      cse/sip/latest       cse/vtk/latest
cse/hdf5/latest      cse/pypcap/1.1         cse/smartgit/1.5.5   cse/vtk_mesa/5.4.2
cse/libpcap/1.0      cse/pypcap/latest      cse/smartgit/latest  cse/vtk_mesa/5.6.0
cse/libpcap/latest   cse/pyqt/4.6.1         cse/sqlite/3.6.22    cse/vtk_mesa/latest
cse/libxml2/2.7.4    cse/pyqt/latest        cse/sqlite/latest    cse/xdmf/2009.12.01
cse/libxml2/latest   cse/pyro/3.6           cse/swig/1.3.37      cse/xdmf/latest
cse/mesa/7.0.4       cse/pyro/latest        cse/swig/2.0.0       cse/xdmf_mesa/2009.12.01
cse/mesa/7.7.1       cse/python/2.6.4       cse/swig/latest
cse/mesa/latest      cse/python/latest      cse/tau/1.0
cse/mpi4py/1.2.1     cse/qt/4.5.2           cse/tau/latest

---------------------- /usr/cta/CSE/Analysis/modules/rel-tools ----------------------------
cse-analysis/gnuplot/4.4.0       cse-analysis/matplotlib/latest   cse-analysis/scipy/0.7.0
cse-analysis/gnuplot/latest      cse-analysis/octave/3.2.4        cse-analysis/scipy/latest
cse-analysis/matplotlib/0.99.1.2 cse-analysis/octave/latest

------------------------ /usr/cta/CSE/Apps/modules/rel-apps -------------------------------
cse/paraview/3.6.2   cse/paraview/latest      cse/paraview_mesa/3.8.0
cse/paraview/3.8.0   cse/paraview_mesa/3.6.2  cse/paraview_mesa/latest

--------------------------- /usr/cta/CSE/modules/rel-apps ---------------------------------
cse/doxygen/1.5.9    cse/openmpi/latest       cse/paraview_mesa/3.8.0  cse-info
cse/doxygen/latest   cse/paraview/3.6.2       cse/paraview_mesa/latest cse-tools
cse/graphviz/2.22.2  cse/paraview/3.8.0       cse/xframework/2008.10
cse/graphviz/latest  cse/paraview/latest      cse/xframework/latest
cse/openmpi/1.4.1    cse/paraview_mesa/3.6.2  cse-beta

----------------------------- /usr/cta/CSE/modules/COTS -----------------------------------
cseinit       cseinit-devel

Load additional module:

To load the latest version of ParaView, use:

module load cse/paraview
or
module load cse/paraview/latest
or
module load cse/paraview/3.8.0

All of these are valid ways to load the latest version of ParaView. Since not all packages are available on all systems, you can get the most up-to-date information on each system by using the "module avail" command.

To load the previous version of ParaView, you must reference the version number. You would use the command:

module load cse/paraview/3.6.2

Module load commands can be combined if you want to load multiple packages at the same time. For example, if you want to load cseinit, cse-tools, ParaView, and Cmake, you could type:

module load cseinit cse-tools cse/paraview cse/cmake

Loading additional modules will add the appropriate environment variables to allow the packages to run. If you want to know where the actual programs are located, look at your $PATH environment variable by running "echo $PATH".

4. List of Applications

The current list of CSE applications are:

Paraview
Doxygen
Graphviz
XframeWork
Visit

4.1. Running ParaView Client/Server

There is a pdf at the following link with the instructions for doing this: Simplified SSH Tunnels for ParaView Client/Server

4.2. Running Visit

4.2.1. Target User

These instructions are intended for Mac, Windows or Linux users outside of ARL which do not receive the standard rdist of Visit. Users that receive the ARL standard rdist have the host profiles bundled with Visit automatically and do not need to copy any host profiles into their account.

Users outside ARL should copy the host XML files from the HPC system (Harold) from /usr/cta/visit/version/current/.visit/hosts directory to their local workstation.

4.2.2. Initializing VisIt Host Profiles

Copy the xml host files to the hosts directory

Windows: \Users\username\Documents\VisIt 2.0.1\hosts
Linux: ~/.visit/hosts
Start VisIt
Go to Options --> Host Profiles
For each host you added, click on it and change the Username to your username on the remote host and click "Apply".

Also, under Launch Profiles --> Parallel, change "Default Bank / Account" to your Project Account number on the HPC machine.
Click Dismiss
To save the options for future use:
Options --> Save Options

4.2.3. Starting a Remote Session

Get a kerberos ticket
Start VisIt
File --> Open
Under Host, choose the HPC resource you want to connect to.

This will open a connection to the remote resource

Choose the file you want, then click "OK".

4.2.4. Remote Rendering Settings

Screenshot of VisIt

5. List of Tools

The current list of CSE tools includes:

cmake
django
django_json
dojango
dojo
flup
freealut
git
hdf5
imaging
libpcap
libxml2
mesa
mpi4py
netdmf
netdmf_editor
numpy
octave
openal
protovis
pypcap
pyqt
pyro
python
python-jsonrpc
qt
scipy
scons
sip
swig
tau
tcltk
valgrind
vtk
xdmf
xframework
openmpi (on some machines)

6. Downloading and Building CSE

6.1. Downloading from the repository

In order to build CSE you will need a working copy of CMake. This can be downloaded from Kitware Inc. at the following URL:

http://www.cmake.org/cmake/resources/software.html

After CMake has been downloaded:

tar -zxvf cmake-2.8.2.tar.gz
cd cmake-2.8.2
./configure --prefix=${HOME}/cmake
make install

To add CMake to your path:

export PATH=${PATH}:${HOME}/cmake/bin ## For Bash
setenv PATH ${PATH}:${HOME}/cmake/bin ## For CSH and TCSH

The base version of CSE can be checked out of the Git repository via the following git command:

git clone http://public.kitware.com/cse.git

To Build and Install CSE:

mkdir build_stable
cd build_stable
cmake -DCSE_INSTALL_PREFIX=${HOME}/CSE ../cse
make

6.2. Downloading Tarball

If for some reason you are not able to download directly from the repository, arrangements can be made with the CSE team to provide you with tarball to download from an FTP site for installation. Once you have contacted the CSE team, you will receive an email with directions on how to download the tarball.