[[PageOutline]] = Application mapping = The QCG-Computing service allows to use abstract application names that are automatically mapped to the absolute paths of the wrapper scripts. The mapping is stored in the file: {{{ /etc/qcg/qcg-comp/application_mapfile }}} The file has the following syntax: {{{ APPLICATION-NAME VERSION SCRIPT-PATH }}} where `VERSION` can be an asterix (`*`) which means "any version". The mapping file is periodically reloaded (5 minutes by default), so there is no need to restart the `qcg-compd` service after updating the file. Example `application_mapfile` file: {{{ MATLAB * /opt/qcg-app-scripts/apps/matlab.app NAMD 6.2 /opt/qcg-app-scripts/apps/namd-6.2.app bash * /opt/qcg-app-scripts/apps/bash.app R * /opt/qcg-app-scripts/apps/R.app }}} The scripts for the most common applications are provided by the `qcg-appscripts` RPM package. '''IMPORTANT:''' For the correct work QCG-Computing requires the mapping for the **bash** application, which is mandatory i.e.: {{{ bash * /opt/qcg-app-scripts/apps/bash.app }}} Frequently Asked Question: Can I use simply mapping to `/bin/bash` {{{ bash * /bin/bash }}} ? The answer is "No". You must use the `bash.qcg` which does some additional work (e.g. setup environment variables like QCG_NODEFILE). = `qcg-appscripts` package = The ''qcg-appscripts'' package contains QCG application scripts for the most common applications. Role of the application scripts, apart from launching the proper application, is as follows: * loading the appropriate modules, * converting input files to UNIX character encoding, * setup environment, * launching user's helper scripts - preprocess, postprocess, assistent, * monitoring the execution of application through defined schemes or user scripts, * handling interactive jobs, * etc. All files are installed in the directory: {{{ /usr/share/qcg-appscripts }}} The directory contains the following sub-directories: - apps - application configuration files, - app-scripts - application scripts, - config - QCG application scripts configuration, - core - QCG scripts library, - tools - QCG tools used by applications. The administrator must create/edit application configuration files (apps) that contain specific to each cluster settings, such as: - name of the application module, - environment variables needed by application. Files from `apps` directory should be referenced by QCG-Computing application mapping file (/etc/qcg/qcg-comp/application_mapfile). The application scripts must be accessible by jobs running on cluster worker nodes. Thus directories `apps`, `app-scripts`, `config`, `core` and `tools` must be copied to directory shared by all worker nodes. This is done manually by the administrator by running: {{{ /usr/sbin/qcg-appscripts-deploy }}} script included in the package. The script reads: {{{ /etc/qcg/qcg-comp/app-scripts/config }}} configuration file for destination directory, where scripts should be deployed. == Installation == To install QCG application scripts: 1. install (as root) the `qcg-appscripts` RPM package {{{ yum install qcg-appscripts }}} 2. edit `/etc/qcg/qcg-comp/app-scripts/config` configuration file and set `cluster_shared_path` variable to point to a directory shared by all worker nodes 3. launch the `qcg-appscripts-deploy` script 4. edit/create application configuration files in `$cluster_shared_path/apps` directory and refer to them in the QCG-Computing application mapping file (`/etc/qcg/qcg-comp/application_mapfile`) == Upgrade == After installation of new QCG application scripts package: 1. launch `qcg-appscripts-deploy` script to update files in shared directory. '''IMPORTANT:''' `qcg-appscripts-deploy` updates directories `app-scripts`, `core`, `tools`. `apps` directories (besides `bash.app`, `common.app` and `template.app`) are not overwritten. = New application = To handle a application, the following elements must be provided: * application configuration file (script in `apps` directory), * application script (script in `app-scripts` directory), * mapping in `/etc/qcg/qcg-comp/application_mapfile` file. == Application configuration file == The application configuration file is a simple script that: * loads the required modules or sets PATH/LD_LIBRARY_PATH variables, * calls proper application script (from the `app-scripts` directory). To create a new application configuration the `apps/template.app` file can be used as a base. Example application configuration file for NAMD application: {{{ #!/bin/bash . $( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/common.app module load namd/current MPIEXEC=charmrun MPIEXEC_ARGS="++verbose ++local +p`cat $QCG_NODEFILE | wc -l`" . $QCG_APP_ROOT_DIR/app-scripts/namd.qcg }}} == Application script == The application script is responsible for: * validation of input arguments, * preparation of input data, * launching the application with proper parameters, * optional post processing of output data/files. Example application script file for NAMD application: {{{ #!/bin/bash QCG_APP_NAME="namd" . $QCG_APP_ROOT_DIR/core/core.qcg if [ $# -ne 1 ] then echo "Illegal number of arguments: $#" echo "Usage:" echo "namd.qcg NAMD-FILE" exit 1 fi #QCG -> NAMD MAIN_FILE="$1" NAMD_PATH=`which namd2` qcg_log_debug "main file = $MAIN_FILE " qcg_log_debug "using NAMD = $NAMD_PATH " qcg_app_wrapper ${MPIEXEC:-mpiexec} $MPIEXEC_ARGS $NAMD_PATH $MAIN_FILE }}} Every application script has to include the `core.qcg` script which provides rich function library that can be used by application scripts. In the above simple example, the application script only validates number of arguments and uses the `qcg_app_wrapper` function to launch MPI version of the NAMD application. The `qcg_app_wrapper` function should always be used to launch application, as it also activates other QCG mechanisms such as: user's pre-processing & post-processing scripts, application monitoring, output file notifications, mounting remote file systems etc. Any modification/conversion of input files should be done in the `qcg_pre_callback` function, for example: {{{ function qcg_pre_callback { if [ -e "$BASH_SCRIPT" ]; then dos2unix $BASH_SCRIPT 2>> qcg.debug fi } if [ "$BASH_SCRIPT_ARGS" ]; then qcg_app_wrapper /bin/bash '$BASH_ARGS' -c \"'$BASH_SCRIPT $BASH_SCRIPT_ARGS'\" else qcg_app_wrapper /bin/bash '$BASH_ARGS' -c '$BASH_SCRIPT' fi }}} = New application version = To create a new version of the already supported application, it is sufficient: 1. to copy the existing application configuration file, eg: {{{ cp /opt/qcg-app-scripts/apps/namd.app /opt/qcg-app-scripts/apps/namd-6.2.app }}} 2. to change the module to be loaded and/or environment variables in `$cluster_shared_path/apps/namd-6.2.app` 3. to add a new application version to the QCG application mapping file (`/etc/qcg/qcg-comp/application_mapfile`), eg: {{{ ... NAMD 6.2 /opt/qcg-app-scripts/apps/namd-6.2.app ... }}} = Configuration = == config/config.conf == * QCG_SCRATCH_DIR - the location of job temporary directory (e.g $TMPDIR) - OPTIONAL - default to $TMPDIR (if set) or "/tmp", * QCG_NO_SCRATCH - if not set, application will be launched in QCG_SCRATCH_DIR * QCG_DEBUG - if not set do not create qcg.debug file - OPTIONAL, * MACHINE_FILE - the name of the environment variable pointing to the machine file (e.g. PBS_NODEFILE) - OPTIONAL. * QCG_GROUP_DIR_ROOT - The root of groups home directory (PL-Grid only). Default: PLG_GROUPS_SHARED - OPTIONAL, * QCG_NTF_URL - address of QCG-Notification broker service * PLG_GROUPS_SHARED - same as QCG_GROUP_DIR_ROOT {{{#!comment = Application Scripts - Developers view = This section is target at QCG developers rather than administrators, its list environment variables which influence job life cycle: == Common Input Environment Variables == * QCG_MODULES_LIST - list of environment modules to be loaded (separated by spaces) * QCG_PREPROCESS - file name of the pre process script * QCG_MONITOR - file name of the monitor script * QCG_MONITOR_INTERVAL - determines how frequently the QCG_MONITOR script should be called (in seconds). - TBD * QCG_NTF_CONSUMER_URL - the address of the Notification consumer interested in receiving notification about job output status change - TBD * QCG_NTF_WATCH_PATTERN - the regular expression that should trigger notification - TBD * QCG_NTF_WATCH_FILE - the file to be watched (default: stdout/err file) - TBD * QCG_NTF_WATCH_INTERVAL - determines how often the watched file should be checked (in seconds) - TBD * QCG_POSTPROCESS - file name of the post process script * QCG_OUTERR_FILE - target filename of the joined output/error stream file (default: output.log) * QCG_ZIPPED_INPUTS - set if the input files must be unziped first (e.g. QCG_ZIPPED_INPUTS=inputs.zip) * QCG_ZIP_OUTPUTS - whether to zip results into $QCG_ZIP_OUTPUTS.zip file * QCG_ZIP_OUTPUTS_FILTER - The wildcard pattern of files to be stored in the zip file (TBD) * QCG_COMP_PROCESSES_MAP - topology of hybrid application (e.g. QCG_COMP_PROCESSES_MAP="4:1:1:1") * QCG_FORCE_SCRATCH - force chdir to temporary directory = Application Scripts - Users view = Here we list environment variables available in any job submitted as QCG-Application (e.g. bash): * QCG_NODEFILE - the machinefile like file with list of nodes allocated for the job * QCG_PPN - number of processes per node * QCG_SPN - number of allocated slots per node (usually equal to QCG_PPN, differ only if job was submitted with: #QCG node=a:b:c where b>c) * QCG_PROCS - total number of proceses }}}