Skip Nav

Computational Science Environment (CSE)

Table of Contents

1. Introduction

CSE Logo In efforts to provide a common platform for utilizing and developing data analysis and assessment applications in heterogeneous High Performance Computing (HPC) environments, the Computational Science Environment (CSE) has been created. CSE is a module based software release structure for the software platform once known as the Interdisciplinary Computing Environment (ICE).

CSE provides a stable suite of data analysis and assessment tools, applications and libraries that are common across most HPC environments and developers workstations. CSE includes the eXtensible Data Model and Format (XDMF), a common data hub where HPC codes and tools can efficiently exchange data values and meaning. CSE consists of many tools like Python, VTK, and ParaView that can assist with day-to-day visualization requirements. CSE provides a common platform that enables the development of full featured portable HPC applications.

1.1. Systems with CSE Installed

Systems with CSE Installed
LocationSystemInstall Directory
ARL Harold /usr/cta/CSE
Pershing /usr/cta/CSE
Hercules /usr/cta/CSE
MRAP /usr/cta/CSE
ABUtility /app/CSE/CSE
CABUtility /app/CSE/CSE
thufir /panfs/panfs/home/CSE
Navy Haise /site/cse/CSE
Kilrain /site/cse/CSE
AFRL Spirit /apps/cse/CSE
ERDC Garnet /usr/local/usp/cseb_cse/CSE

1.2. Contact Information

Support E-Mail: usarmy.apg.arl.list.cse-support@mail.mil.

2. Building CSE

A script to build CSE in an automated fashion is located at cse_stable/Tools/cseBuild.sh.

~$ cse_stable/Tools/cseBuild.sh [Flag Option] [Flag Argument]

The script relies on the repository code to be located under $HOME/Development unless otherwise specified.

/home/username/Development
~$ ls
cse_analysis.git  cse_datamodels.git  cse_stable.git

If the code cannot be located, the script will automatically download and setup the repositories.

When pulling the code from outside of ARL, is it important to specify use of the Kitware repository with the flag option "-r kitware" added to the command line.

~$ cseBuild.sh -r kitware

Use the flag option "-i" to specify the directory to install CSE.

~$ cseBuild.sh -i $HOME/Apps/CSE

If you cannot or do not wish the pull the latest repository you should use the "nopull" option.

~$ cseBuild.sh -o nopull -i $HOME/Apps/CSE

If you want to control the load average of the build, so that is does not take up too much of your system's resources, use the flag option "-l".

~$ cseBuild.sh -l4

In this case the load average of the build will not exceed 4. Also, since the install directory has not been specified, CSE will install to the default location.

For any additional options and information, please reference the help documentation by using the ".help" option.

~$ cseBuild.sh --help

3. Using CSE

3.1. Sourcing Module

If your system does not have the module program sourced, you may source the CSE version.

~$ source [CSE_INSTALL_DIR]/modules/init/bash

You can replace bash with whatever linux shell you are using. Alternatives currently provided include csh, ksh, perl, python, sh, tcsh, and zsh. Once you have sourced this directory, you should have access to the command 'modulecmd' and an alias for 'module' in your environment. Also, your modules path should be set up to point to the CSE installation you sourced.

3.2. Module Use

The first step to using CSE is to tell module where to find the cseinit files.

~$ module use [CSE_INSTALL_DIR]/modules/COTS

3.3. Checking Available Modules

~$ module avail

You should see output similar to below.

------------------------ /home/username/CSE/modules/COTS/ ------------------------- 
cseinit       cseinit-devel

3.4. Module Loading and Unloading

Once CSE is installed and modules are sourced, you need to load cseinit. The cseinit module will set up your environment to see all of the available packages in CSE.

~$ module load cseinit

Loading the cseinit module will allow you to use the full extent of CSE. Call the avail command again to see all of the loadable modules.

---------------------- /home/username/CSE/Datamodels/modules/data-tools ----------------------- 
cse/dpkt/1.8           cse/libpcap/latest     cse/pybindgen/0.16.0   cse/xdmf/latest 
cse/dpkt/latest        cse/netcdf/4.3.0       cse/pybindgen/latest   cse/xdmf2/2013.10.07 
cse/hdf5/1.8.11        cse/netcdf/latest      cse/pypcap/1.1         cse/xdmf2/latest 
cse/hdf5/latest        cse/netdmf2/2013.10.02 cse/pypcap/latest 
cse/libpcap/1.4.0      cse/netdmf2/latest     cse/xdmf/2011.11.16 

------------------------ /home/username/CSE/Analysis/modules/rel-tools ------------------------ 
cse/blosc/1.1         cse/lapack/latest     cse/numpy/1.7.0       cse/scipy/latest 
cse/blosc/latest      cse/lzo/2.06          cse/numpy/latest      cse/six/1.4.1 
cse/cython/0.19.1     cse/lzo/latest        cse/pygal/1.1.0       cse/six/latest 
cse/cython/latest     cse/matplotlib/1.3.0  cse/pygal/latest      cse/snappy/1.1.0 
cse/dateutil/2.1      cse/matplotlib/latest cse/pyparsing/2.0.1   cse/snappy/latest 
cse/dateutil/latest   cse/nose/1.3.0        cse/pyparsing/latest  cse/tornado/3.1.1 
cse/gnuplot/4.6.3     cse/nose/latest       cse/pytables/3.0.0    cse/tornado/latest 
cse/gnuplot/latest    cse/numexpr/2.2.1     cse/pytables/latest 
cse/lapack/3.4.2      cse/numexpr/latest    cse/scipy/0.12.0 

---------------------------- /home/username/CSE/modules/rel-tools ----------------------------- 
cse/cmake/2.8.11        cse/mesa/7.7.1          cse/python/2.7.5        cse/swig/2.0.11 
cse/cmake/latest        cse/mesa/latest         cse/python/latest       cse/swig/latest 
cse/gccxml/latest       cse/mgen/5.02           cse/python3/3.3.2       cse/tcltk/8.5.14 
cse/gccxml/master       cse/mgen/latest         cse/python3/latest      cse/tcltk/latest 
cse/git/1.8.4           cse/mpi4py/1.3.1        cse/qt/4.8.5            cse/vtk/5.6.1 
cse/git/latest          cse/mpi4py/latest       cse/qt/latest           cse/vtk/6.0.0 
cse/ipython/1.1.0       cse/pygccxml/1.0.0      cse/readline/6.2.4.1    cse/vtk/latest 
cse/ipython/latest      cse/pygccxml/latest     cse/readline/latest     cse/vtk_mesa/5.6.1 
cse/libxml2/2.9.1       cse/Pygments/1.6        cse/scons/2.3.0         cse/vtk_mesa/6.0.0 
cse/libxml2/latest      cse/Pygments/latest     cse/scons/latest        cse/vtk_mesa/latest 
cse/libxslt/1.1.28      cse/pylint/1.0.0        cse/sip/4.15.2          cse/winpdb/1.4.8 
cse/libxslt/latest      cse/pylint/latest       cse/sip/latest          cse/winpdb/latest 
cse/lineprofiler/1.0b3  cse/pyqt/4.10.3         cse/sqlite/3.7.17       cse/wxpython/2.8.12.1 
cse/lineprofiler/latest cse/pyqt/latest         cse/sqlite/latest       cse/wxpython/latest 
cse/lxml/3.2.3          cse/pyro/3.16           cse/statprof/0.1.2 
cse/lxml/latest         cse/pyro/latest         cse/statprof/latest 

----------------------------- /home/username/CSE/modules/rel-apps ----------------------------- 
cse/doxygen/1.8.5   cse/graphviz/2.34.0 cse/openmpi/1.6.5   cse-info 
cse/doxygen/latest  cse/graphviz/latest cse/openmpi/latest  cse-tools 

------------------------------ /home/username/CSE/modules/COTS/ ------------------------------- 
cseinit       cseinit-devel 

When you are done using a module, you may unload it using the module unload command.

4. CSE Structure

CSE is structured into five parts, each with its own repository.

4.1. Stable

Information: Stable contains many development and debugging tools needed for software developers, as well as some of the base components to allow CSE to work on many different systems.

Packages: cmake, curl, doxygen, gccxml, graphviz, ipython, libxml2, libxslt, lineprofiler, lxml, mesa, mgen, modules, mpi4py, openmpi, patch, pygccxml, pygments, pylint, pyqt4, pyro3, python, qt4, readline, scons, setuptools, sip, sqlite, statprof, swig, tcltk, vtk, winpdb, wxpython

4.2. Datamodels

Information: Datamodels contains packages for handling various data formats.

Packages: boost, dpkt, hdf5, libpcap, netcdf, netdmf2, netdmf_plugin, pybindgen, pypcap, xdmf, xdmf2

4.3. Analysis

Information: Analysis contains scientific and visualization tools to help analyze different types of data.

Packages: cython, dateutil, gnuplot, lapack, lzo, matplotlib, nose, numexpr, numpy, pygal, pyparsing, pytables, scipy, six, snappy, tornado

4.4. Visualization

Information: ParaView is an open-source, multi-platform data analysis and visualization application used in a wide variety of scientific fields. VisIt is a free, interactive, parallel visualization and graphical analysis tool for viewing scientific data on Unix and PC platforms.

Packages: paraview, visit

5. Installing Python3 Packages

Most python packages will come with a setup.py file as part of their source code. This makes the build and install process fairly simple and will be the preferred method of this tutorial as we need to be able to set a specific location to install the library instead of installing directly into the python3 directories.

5.1. Load CSE

In order to install a new python package for python3 you need to load the python3 module from CSE. If you do not have module installed to your system you can source it from CSE.

~$ source [CSE_INSTALL_DIRECTORY]/Modules/init/[bash, csh, ksh, perl, python, sh, tcsh, zsh]

Once sourced you can use it to load the CSE init module.

~$ module use [CSE_INSTALL_DIRECTORY]/Modules/COTS
~$ module load cseinit

5.2. Load Python3 Module

First you need to load the CSE Python3 module.

~$ module load cse/python3/latest

Remember the Python3 binary is named .python3. by default and not simply "python".

5.3. Build

Some packages require you to call the build command first.

~$ python3 setup.py build

5.4. Install

Next you need to call the install command. The prefix should be set to a directory you have write access too such as your home directory.

~$ python3 setup.py install --prefix=[INSTALL_DIRECTORY]

Add Install Directory to the PYTHONPATH

~$ PYTHONPATH=[INSTALL_DIRECTORY]/lib/python3.3/site-packages