Guide to the PBS Queuing System
Table of Contents
IntroductionPBS Modules
Basic PBS
Basic PBS Job Commands
Command Translation Table
PBS Environment Variables
PBS Sample Scripts for Common MPI Jobs
Harold/TOW Samples
MJM Samples
MRAP Samples
Contacting CCAC
Links to More Information
Introduction
Batch jobs for MJM, Harold, TOW and MRAP at the ARL DSRC are submitted utilizing the PBS queuing system. The PBS module should be automatically loaded at startup/login.
PBS Modules
Please be sure a pbs module is loaded before running any PBS scripts or commands. You can test this by executing a "module list" :
module list Currently Loaded Modulefiles: 1) modules 2) lsf 3) Master 4) pbs
One can see that module 4 above is the pbs module and has been loaded. If you do not see this, then issue a module load of pbs like this:
module load pbs
Basic PBS
Although PBS has many commands and settings, knowing a basic few of those can allow most anyone to run most jobs.
PBS directives can be embedded in submission scripts by prefixing a script line with "#PBS". A few notable PBS directives are shown below.
To set the total walltime to run your job:
#PBS -l walltime=xx:yy:zz
To set the queue:
#PBS -q queue_name
Where queue name is usually debug, standard, challenge or background. Other queues are available but they are reserved for special cases and/or staff. Sample:
#PBS -q queue_name
To pass all the environmental variables to the parallel jobs, the following PBS directive must be present.
#PBS -V
To select the total amount of nodes (not cores!), x, and the amount of cores to use per node, y, use this PBS directive:
#PBS -l select=x:ncpus=8:mpiprocs=y
The total amount of cores will be equal to x*y (note that "ncpus=" will always be set to 8 for Harold & TOW, but "ncpus=" must be set to 4 for MJM), example:
#PBS -l select=4:ncpus=8:mpiprocs=8
This will select 32 cores (4*8=32) for a computational run. Note that if one wishes to use all the memory per node for one MPI task (about 24 GB on Harold), the select can be written:
#PBS -l select=32:ncpus=8:mpiprocs=1
For those familiar with our previous LSF system, mpiprocs behaves just like the old "ptile=" LSF statement.
You must also add the below PBS directives:
# Keep the standard out/error files #PBS -k oe # Don't restart job if it fails #PBS -r n
On the MRAP system, to select the amount of cores, you must use this format:
#PBS -l mppwidth=16 #PBS -l mppnppn=8
Where mppwidth is the total amount of cores your job will utilize and mppnppn is the amount of cores/node to use. The example above will run a 16 cpu jobs on two nodes (because MRAP has 8 cores/node). For those familiar with our previous LSF system, mppnppn behaves just like the old "ptile=" LSF statement.
To set the name of the job:
#PBS -N JOB_NAME
Example:
#PBS -N MPI_JOB_93
Please be sure not to have any spaces or non-alphanumeric characters in JOB_NAME with the exception of the underscore!
The standard output and standard error output from your job can be redirected into one file (optional):
#PBS -j oe
And will be named JOB_NAME.<job #> where JOB_NAME is from the "#PBS -N" directive and <job #> is given by PBS when PBS runs your job.
And to set your project identifier:
#Set your project alphanumeric (may look like ARLAP96090RAY) #PBS -A xxxxxxxxxxx
Basic PBS Job Commands
To submit a job to the PBS queue type:
qsub pbs_script.csh
To monitor your jobs, enter:
qstat -sw -u`whoami`
To kill a job enter:
qdel <job number>
To see which nodes are being utilized for a job id:
qstat - n <job number>
To see the free blades/nodes not in use:
pbsnodes -a |grep -B 3 "state = free" | grep Mom | awk '{print $3}'
On the MRAP cluster only to view all jobs and their respective core allocation type:
apstat -a
More information is available from the man pages
man qsub
Command Translation Table
The tables below show how to translate LSF commands, features, and environment variables into their PBS counterparts. The tables below do NOT list all LSF and PBS commands and features. There are commands, features, and env variables in each queuing system that have no counterpart in the other.
| Command | Grid Engine | LSF | PBS |
|---|---|---|---|
| Submit a job | qsub script | bsub < script | qsub |
| Check Job Status | qstat | bjobs, bhist | qstat -[asnf] |
| Job Deletion | qdel | bkill | qdel |
| Modify Job Submission | qalter | bmod, bswitch | qalter |
| Hold or Suspend | qhold | bstop | qhold |
| Release or Unsuspend | qrls | bresume | qrls |
| Accounting | qacct | bacct | tracejob |
| Host Status | qhost | bhosts | pbsnodes |
| Queue Configuration | qconf -sq or -sql | bqueues | qstat -q |
| Host Configuration | qconf -se or -sel | bmgroup | qmgr |
| Other Configuration | qconf | bugroup, busers | - |
| Job Submission | Grid Engine | LSF | PBS |
|---|---|---|---|
| Command | qsub script | bsub < script | qsub script |
| Option prefix | #$ | #BSUB | #PBS |
| Wallclock | -l 48hr | -W 48:00 | -l walltime=hh:mm:ss |
| Parallel Env | -pe name #CPUs | -n #CPUs -a mpich_gm |
-l select=node:ncpus=x |
| Job Name | -N name | -J name | -N name |
| Output file | -o filename | -o filename | -o filename |
| Error file | -e filename | -e filename | -e filename |
| Project | -P project | -P project | -A project |
| Job dependency | -hold_jid job# | -w expression | -W depend=list |
| Current working directory | -cwd | is default | is default |
PBS Environment Variables
There are many PBS environment variables. One needs only to know a select, few important environment variables to get started as shown below. This table is set up assuming an 8 CORE (2 node) job on the MJM cluster.
| PBS Variable | Typical Usage | Example Output |
LSF Related Variable |
|---|---|---|---|
| PBS_JOBID | echo ${PBS_JOBID} | cut -f1 -d. | 321222 | LSB_JOBID |
| PBS_O_LOGNAME | echo $PBS_O_LOGNAME | userid | LOGNAME |
| PBS_NODEFILE | cat ${PBS_NODEFILE} | sort | node0001 node0001 node0001 node0001 node0253 node0253 node0253 node0253 |
LSB_HOSTS |
| - | echo `wc -l ${PBS_NODEFILE} | cut -f1 -d" " ` | 8 | NSLOTS |
| PBS_ARRAY_INDEX | echo "Now starting JOB ARRAY # ${PBS_ARRAY_INDEX}" | Now starting JOB ARRAY #72 | LSB_JOBINDEX |
Some users might find the additional PBS environment variables listed in the table below useful.
| Variable | Meaning |
|---|---|
| NCPUS | Number of threads, defaulting to number of CPUs, on the vnode. |
| OMP_NUM_THREADS | Same as NCPUS and mainly used for OpenMP. |
| PBS_ARRAY_ID | Identifier for job arrays. Consists of sequence number. |
| PBS_ARRAY_INDEX | Index number of subjob in job array. |
| PBS_ENVIRONMENT | Indicates job type: PBS_BATCH or PBS_INTERACTIVE |
| PBS_JOBCOOKIE | Unique identifier for inter-MOM job-based communication. |
| PBS_JOBID | The job identifier assigned to the job or job array by the batch system. |
| PBS_JOBDIR | Pathname of job-specific staging and execution directory. |
| PBS_JOBNAME | The job name supplied by the user. |
| PBS_MOMPORT | Port number on which this job's MOMs willcommunicate. |
| PBS_NODEFILE | The filename containing a list of vnodes assigned to the job. |
| PBS_NODENUM | Logical vnode number of this vnode allocated to the job. |
| PBS_O_HOME | Value of HOME from submission environment. |
| PBS_O_HOST | The host name on which the qsub command was executed. |
| PBS_O_LANG | Value of LANG from submission environment. |
| PBS_O_LOGNAME | Value of LOGNAME from submission environment. |
| PBS_O_MAIL | Value of MAIL from submission environment. |
| PBS_O_PATH | Value of PATH from submission environment. |
| PBS_O_QUEUE | The original queue name to which the job was submitted. |
| PBS_O_SHELL | Value of SHELL from submission environment. |
| PBS_O_SYSTEM | The operating system name where qsub was executed. |
| PBS_O_TZ | Value of TZ from submission environment. |
| PBS_O_WORKDIR | The absolute path of directory where qsub was executed. |
| PBS_QUEUE | The name of the queue from which the job is executed. |
| PBS_TASKNUM | The task (process) number for the job on this vnode. |
| TMPDIR | The job-specific temporary directory for this job. |
PBS Sample Scripts for Common MPI Jobs
Shown below are various sample PBS scripts for common MPI executables. It should be noted that these sample scripts should also exist in /usr/cta/SCR/MPI* area on each machine.
Harold/TOW Samples
A sample PBS script to run a 128 core (16 nodes) MPI program compiled with Intel Fortran and utilizing the SGI-MPI library on HAROLD or TOW systems.
#!/bin/csh
#
# Sample PBS script to run an MPI program compiled with Intel
# Fortran and SGI-MPI
#
#PBS -l walltime=01:00:00
# 128 cpu run using 8 cpus/node
# select = # of nodes
# ncpus is ALWAYS set to 8!
# mpiprocs is the amount of cpus on each node to use
# This run will use (select)x(mpiprocs) cpus = 16*8=128 cpus
#PBS -l select=16:ncpus=8:mpiprocs=8
# scatter and exclusive mode access
#PBS -l place=scatter:excl
#set name of job (do not use any spaces!)
#PBS -N MPI_JOB
# set queue: debug, standard, challenge, background
#PBS -q standard
# Keep the standard out/error files
#PBS -k oe
# Join the standard error/out into one file (optional)
#xPBS -j oe
# Don't restart job if it fails
#PBS -r n
# Pass all environment variables to MPI jobs
#PBS -V
#Set your project alphanumeric (may look like ARLAP96090RAY)
#PBS -A xxxxxxxxxxx
#
#############################################################
# change the below input variable to your own filename
#############################################################
set input=picalc.exe
#
#
# PBS to LSF environmental variable equivalence settings
#
setenv JOBID `echo ${PBS_JOBID} | cut -f1 -d.`
echo "JOBID,PBS_JOBID=$JOBID,$PBS_JOBID"
setenv LOGNAME ${PBS_O_LOGNAME}
echo "LOGNAME,PBS_O_LOGNAME=$LOGNAME,$PBS_O_LOGNAME"
setenv NSLOTS "`wc -l ${PBS_NODEFILE}| cut -f1 -d' '`"
echo "**** NSLOTS=${NSLOTS}"
#
echo MPI job $JOBID starting at `date` on `hostname`
echo starting in `pwd`
#
# exports or setenv
#
setenv JOBID `echo $PBS_JOBID | awk -F'.' '{print $1}'`
#setenv MPI_DSM_DISTRIBUTE
# For OpenMP jobs only! (NOT OPENMPI)
#setenv OMP_NUM_THREADS 1
## Get the execution host name
set exechost=`hostname -s`
echo exechost is $exechost
set jobhost=`uname -n`
set outfile=out.dat
set TMPD=/usr/var/tmp/$LOGNAME/$JOBID
set TMPC=/usr/var/tmp/$LOGNAME
if (! -e $TMPD ) mkdir -p $TMPD
#
# Copy Files to $TMPD which is usually /usr/var/tmp/userid/job #
#
cp ${input} ${TMPD}
if ( $status != 0 ) then
# Can't find input in $HOME area, then try /usr/var/tmp/userid/ area
echo "Cannot find $input in `pwd` area, trying ${TMPC} area..."
cd ${TMPC}
cp ${input} ${TMPD} || ( echo "*ERROR! Cannot cp ${input} ${TMPD} !" ; exit 199 )
endif
cd $TMPD
ls -ltr
unlimit
module load compiler/intel11.0 mpi/sgi_mpi-1.23
#
# REMEMBER! The value after "-np" must be equal to (select)*(mpiprocs)
# in the "PBS -l select=" statement above. In this case it is 128.
#
mpiexec_mpt -np 128 -v ./picalc.exe >& picalc.out
set st=$status
echo "MPI program ended with status $st on `date`"
exit $st
A sample job script for a JOB ARRAY in PBS on the HAROLD system is:
#!/bin/csh # # Sample PBS script to run an MPI program compiled with Intel # Fortran and SGI-MPI # #PBS -l walltime=01:00:00# Job array from 1 to 10 in steps of 1 #PBS -J 1-10:1# This run will use (select)x(mpiprocs) cpus = 16*8=128 cpus #PBS -l select=16:ncpus=8:mpiprocs=8 # scatter and exclusive mode access #PBS -l place=scatter:excl #set name of job (do not use any spaces!) #PBS -N MPI_JOB # set queue: debug, standard, challenge, background #PBS -q standard # Keep the standard out/error files #PBS -k oe # Join the standard error/out into one file (optional) #xPBS -j oe # Don't restart job if it fails #PBS -r n # Pass all environment variables to MPI jobs #PBS -V #Set your project alphanumeric (may look like ARLAP96090RAY) #PBS -A xxxxxxxxxxx # ############################################################# # change the below input variable to your own filename ############################################################# set input=picalc.exe #echo "Now starting JOB ARRAY # ${PBS_ARRAY_INDEX} at `date`"# # Environmental variable equivalence settings # setenv JOBID `echo ${PBS_JOBID} | cut -f1 -d.` echo "JOBID,PBS_JOBID=$JOBID,$PBS_JOBID" setenv HOSTS `cat ${PBS_NODEFILE} | tr ' ' '\n' ` echo "HOSTS,PBS_NODEFILE=$HOSTS,$PBS_NODEFILE" setenv LOGNAME ${PBS_O_LOGNAME} echo "LOGNAME,PBS_O_LOGNAME=$LOGNAME,$PBS_O_LOGNAME" setenv NSLOTS "`wc -l ${PBS_NODEFILE}| cut -f1 -d' '`" echo "**** NSLOTS=${NSLOTS}" # echo MPI job $JOBID starting at `date` on `hostname` echo starting in `pwd` # # exports or setenv # setenv JOBID `echo $PBS_JOBID | awk -F'.' '{print $1}'` ## Get the execution host name set exechost=`hostname -s` echo exechost is $exechost set jobhost=`uname -n` set outfile=out.dat set TMPD=/usr/var/tmp/$LOGNAME/$JOBID set TMPC=/usr/var/tmp/$LOGNAME if (! -e $TMPD ) mkdir -p $TMPD # # Copy Files to $TMPD which is usually /usr/var/tmp/userid/job # # cp ${input} ${TMPD} if ( $status != 0 ) then # Can't find input in $HOME area, then try /usr/var/tmp/userid/ area echo "Cannot find $input in `pwd` area, trying ${TMPC} area..." cd ${TMPC} cp ${input} ${TMPD} || ( echo "*ERROR! Cannot cp ${input} ${TMPD} !" ; exit 199 ) endif cd $TMPD ls -ltr unlimit module load compiler/intel11.0 mpi/sgi_mpi-1.23 # # REMEMBER! The value after "-n" must be equal to (select)*(mpiprocs) # in the "PBS -l select=" statement above. In this case it is 32. # mpiexec_mpt -np 128 ./picalc.exe >& picalc.out set st=$status echo "MPI program ended with status $st on `date`" exit $st
MJM Samples
A sample PBS script to run an 8 core (2 node) MPI program on MJM compiled with Intel Fortran and utilizing the OPEN-MPI library. This script should be available for one to copy in the /usr/cta/SCR/MPI directory.
#!/bin/csh
#
# Sample PBS script to run an MPI program compiled with Intel
# Fortran and OpenMPI
#
#PBS -l walltime=01:00:00
# 8 core run using 4 cores/node
# select = # of nodes
# ncores is ALWAYS set to 4!
# mpiprocs is the amount of cores on each node to use
#PBS -l select=2:ncpus=4:mpiprocs=4
# scatter and exclusive mode access
#PBS -l place=scatter:excl
#set name of job (do not use any spaces!)
#PBS -N MPI_JOB
# set queue: debug, standard, challenge, background
#PBS -q standard
# Keep the standard out/error files
#PBS -k oe
# Join the standard error/out into one file (optional)
#xPBS -j oe
# Don't restart job if it fails
#PBS -r n
# Pass all environment variables to MPI jobs
#PBS -V
#Set your project alphanumeric (may look like ARLAP96090RAY)
#PBS -A xxxxxxxxxxx
#
#############################################################
# change the below input variable to your own filename
#############################################################
set input=picalc.exe
#
#
# Environmental variable equivalence settings
#
setenv JOBID `echo ${PBS_JOBID} | cut -f1 -d.`
echo "JOBID,PBS_JOBID=$JOBID,$PBS_JOBID"
setenv HOSTS `cat ${PBS_NODEFILE} | tr ' ' '\n' `
echo "HOSTS,PBS_NODEFILE=$HOSTS,$PBS_NODEFILE"
setenv LOGNAME ${PBS_O_LOGNAME}
echo "LOGNAME,PBS_O_LOGNAME=$LOGNAME,$PBS_O_LOGNAME"
setenv NSLOTS "`wc -l ${PBS_NODEFILE} | cut -f1 -d' '`"
echo "**** NSLOTS=${NSLOTS}"
#
echo MPI job $JOBID starting at `date` on `hostname`
echo starting in `pwd`
#
# exports or setenv
#
setenv JOBID `echo $PBS_JOBID | awk -F'.' '{print $1}'`
## Get the execution host name
set exechost=`hostname -s`
echo exechost is $exechost
set jobhost=`uname -n`
set outfile=out.dat
set TMPD=/usr/var/tmp/$LOGNAME/$JOBID
set TMPC=/usr/var/tmp/$LOGNAME
if (! -e $TMPD ) mkdir -p $TMPD
#
# Copy Files to $TMPD which is usually /usr/var/tmp/userid/job #
#
cp ${input} ${TMPD}
if ( $status != 0 ) then
# Can't find input in $HOME area, then try /usr/var/tmp/userid/ area
echo "Cannot find $input in `pwd` area, trying ${TMPC} area..."
cd ${TMPC}
cp ${input} ${TMPD} || ( echo "*ERROR! Cannot cp ${input} ${TMPD} !" ; exit 199 )
endif
cd $TMPD
ls -ltr
unlimit
module load intel
openmpirun.pbs ./picalc.exe >& picalc.out
set st=$status
echo "MPI program ended with status $st on `date`"
exit $st
A sample job script for a JOB ARRAY in PBS on the MJM system is:
#!/bin/csh # # Sample PBS script to run an MPI program compiled with Intel # Fortran and SGI-MPI # #PBS -l walltime=01:00:00 # 8 core run using 4 cores/node # select = # of nodes # ncores is ALWAYS set to 4! # mpiprocs is the amount of cores on each node to use #PBS -l select=2:ncpus=4:mpiprocs=4 #PBS -N MPI_JOB #PBS -q debug# Job array from 1 to 10 in steps of 1 #PBS -J 1-10:1#PBS -V #Set your project alphanumeric (may look like ARLAP96090RAY) #PBS -A xxxxxxxxxxx # ############################################################# # change the below input variable to your own filename ############################################################# set input=picalc.exe #echo "Now starting JOB ARRAY # ${PBS_ARRAY_INDEX} at `date`"# # Environmental variable equivalence settings # setenv JOBID `echo ${PBS_JOBID} | cut -f1 -d.` echo "JOBID,PBS_JOBID=$JOBID,$PBS_JOBID" setenv HOSTS `cat ${PBS_NODEFILE} | tr ' ' '\n' ` echo "HOSTS,PBS_NODEFILE=$HOSTS,$PBS_NODEFILE" setenv LOGNAME ${PBS_O_LOGNAME} echo "LOGNAME,PBS_O_LOGNAME=$LOGNAME,$PBS_O_LOGNAME" setenv NSLOTS "`wc -l ${PBS_NODEFILE}`" echo "**** NSLOTS=${NSLOTS}" # echo MPI job $JOBID starting at `date` on `hostname` echo starting in `pwd` # # exports or setenv # setenv JOBID `echo $PBS_JOBID | awk -F'.' '{print $1}'` ## Get the execution host name set exechost=`hostname -s` echo exechost is $exechost set jobhost=`uname -n` set outfile=out.dat set TMPD=/usr/var/tmp/$LOGNAME/$JOBID set TMPC=/usr/var/tmp/$LOGNAME if (! -e $TMPD ) mkdir -p $TMPD # # Copy Files to $TMPD which is usually /usr/var/tmp/userid/job # # cp ${input} ${TMPD} if ( $status != 0 ) then # Can't find input in $HOME area, then try /usr/var/tmp/userid/ area echo "Cannot find $input in `pwd` area, trying ${TMPC} area..." cd ${TMPC} cp ${input} ${TMPD} || ( echo "*ERROR! Cannot cp ${input} ${TMPD} !" ; exit 199 ) endif cd $TMPD ls -ltr unlimit module load compiler/intel11.0 mpi/sgi_mpi-1.23 openmpirun.pbs ./picalc.exe >& picalc.out set st=$status echo "MPI program ended with status $st on `date`" exit $st
MRAP Samples
A sample PBS script for running a 16 processor MPI job on MRAP:
#!/bin/csh
# Set number of cores to utilize in this run
#PBS -l mppwidth=16
# Set number of cores per node
#PBS -l mppnppn=8
#Set max. time of the run
#PBS -l walltime=00:30:00
#PBS -q debug
# Don't use quotes about JOB_NAME or spaces
#PBS -N JOB_NAME_1
#PBS -j oe
# Pass environment variables to mpi job
#PBS -V
#Set your project alphanumeric (may look like ARLAP96090RAY)
#PBS -A xxxxxxxxxxx
#
cd $PBS_O_WORKDIR
echo "PBS_O_WORKDIR is set to $PBS_O_WORKDIR"
set TMP=`echo ${PBS_JOBID} | cut -f1 -d.`
set TMPD=$PBS_O_WORKDIR/${TMP}
if ( ! -d ${TMPD} ) then
mkdir -p $TMPD
endif
cp ./picalc.exe $TMPD || echo "****ERROR! CANNOT cp ./picalc.exe $TMPD "
cd $TMPD
#setenv PAT_RT_SUMMARY 0
echo "Modules in use: --------------vvvvv"
module list
echo "---------------------^^^^^^^^^^^^^^"
echo "Starting at `date` in `pwd` on `hostname`"
aprun -n 16 -N 8 ./picalc.exe
set st=$status
echo "Done at `date` with status $st"
exit $st
A sample serial job script for PBS is:
#!/bin/csh
#PBS -l mppwidth=16
# Set number of cores per node
#PBS -l mppnppn=8
#PBS -l walltime=00:30:00
#PBS -q debug
# Don't use quotes about JOB_NAME
#PBS -N JOB_NAME_1
#PBS -j oe
#PBS -V
#Set your project alphanumeric (may look like ARLAP96090RAY)
#PBS -A xxxxxxxxxxx
#
echo "Now starting at `date`"
cd $PBS_O_WORKDIR
set TMP=`echo ${PBS_JOBID} | cut -f1 -d.`
set TMPD=$PBS_O_WORKDIR/${TMP}
if ( ! -d ${TMPD} ) then
mkdir -p $TMPD
endif
cp ./picalc.exe $TMPD || echo "**** ERROR! CANNOT cp ./picalc.exe $TMPD"
cd $TMPD
echo "Modules in use: --------------vvvvv"
module list
echo "---------------------^^^^^^^^^^^^^^"
echo "Starting at `date` in `pwd` on `hostname`"
./picalc.exe
set st=$status
echo "Done at `date` with status $st"
exit $st
A sample job script for a JOB ARRAY in PBS on the MRAP system is:
#!/bin/csh
#PBS -l mppwidth=16
# Set number of cores per node
#PBS -l mppnppn=8
#PBS -l walltime=00:30:00
#PBS -q debug
# Don't use quotes about JOB_NAME
#PBS -N JOB_NAME_1
#PBS -j oe
# Job array from 1 to 10 in steps of 1
#PBS -J 1-10:1
#PBS -V
#Set your project alphanumeric (may look like ARLAP96090RAY)
#PBS -A xxxxxxxxxxx
#
echo "Now starting JOB ARRAY # ${PBS_ARRAY_INDEX} at `date`"
cd $PBS_O_WORKDIR
echo "PBS_O_WORKDIR is set to $PBS_O_WORKDIR"
echo "PBS_JOBID=$PBS_JOBID"
set TMP=`echo "${PBS_JOBID}" | cut -f1 -d. | cut -f1 -d[ `
set TMPD=${PBS_O_WORKDIR}/${TMP}
echo "TMPD=${TMPD}"
if ( ! -d ${TMPD} ) then
mkdir -p $TMPD
endif
cp ./picalc.exe $TMPD || echo "**** ERROR! CANNOT cp ./picalc.exe $TMPD"
cd $TMPD
echo "Modules in use: --------------vvvvv"
module list
echo "---------------------^^^^^^^^^^^^^^"
echo "Starting at `date` in `pwd` on `hostname`"
aprun -n 16 -N 8 ./picalc.exe
set st=$status
echo "Done at `date` with status $st"
exit $st
Customers are reminded to use /usr/var/tmp as their scratch space. Do not use /tmp or /var/tmp!
Contacting CCAC
Questions, comments and suggestions are always welcome. If you have have questions about this guide, or any of the ARL DSRC's assets, please contact the DoD HPCMP's Consolidated Customer Assistance Center in any of the following ways:
- On the Web: CCAC
- Over the phone toll-free: 1-877-222-2039 (1-877-CCAC-039)
- Over the phone locally: 937-255-067
- Through email: Help@ccac.hpc.mil
Links to More Information
PBS site Link:
http://www.pbsgridworks.com/
Other Useful Linux Links
Linux High Performance Technical Computing: http://www.linuxhpc.org/
UnixGuide: http://unixguide.net/