The page describes the usage of the QCG-Broker Client, which is the command line text client for the QCG-Broker service.

System pre-requirements

• Installed qcg-broker-client package. The RPM installation procedure for SL5/6 is described here

User Requirements

• User's credential To secure the communication and to authenticate the user to the service the X509 proxy certificate is needed.

Configuration

For its work the client needs user credentials to be present and properly configured. Additionally every user can overwrite some global settings to adapt configuration to its own preferences.

For details please visit the QCG-Client Configuration? page.

QCG Broker Client

The QCG-Client is a java based text client offering access to the QCG infrastructure from the command-line level. The QCG-Client is offered in two versions and the choice of version depends on user's experience and needs:

• QCG-SimpleClient (recommended) is a set of tools (similar to ones offered by queuing systems) for managing of jobs. For user's convenience the simple format of description of jobs (based on #QCG directives) was proposed allowing in a simple way to describe the most popular use cases.
• QCG-AdvancedClient - the client intended for more experienced users requiring more advanced features like workflows, parameter sweep, tasks with many groups of processes with different requirements. The job has to be described by XML document.

QCG Simple Client

The QCG Simple client is recommended for all users who do not need advanced capabilities of the QCG middleware like workflows, parallel jobs with topologies or parameter sweep jobs. In return, it offers access to the most frequently used functionality in very simple and intuitive way. The QCG Simple client consists of a set of command line tools modelled after the set of tools known from popular queueing systems. All tasks submitted to be executed by QCG-Broker must be described in a formal way using the concept of directives known also from queueing systems.

qcg-* Tools

Set of tools allowing to submit/control tasks and to reserve/control resources managed by the QCG infrastructure.

The detailed description of the qcg-* tools?

Submission and controlling of tasks:

•  qcg-cancel - cancel task(s)
•  qcg-clean - clean the working directories of given tasks
•  qcg-connect - connect interactive session to the task
•  qcg-info - display detailed information about the given tasks
•  qcg-list - lists tasks in the system
•  qcg-peek - display ending of (stdout, stderr) streams
•  qcg-proxy - create user proxy certificate
•  qcg-refetch - retry/repeat the transfer of output files/directories
•  qcg-refresh_proxy - refresh user proxy certificate for given tasks
•  qcg-sub - submit the task to be processes by QCG services

Resources reservation and controll:

•  qcg-rcancel - cancel reservation(s)
•  qcg-reserve - reserve resources
•  qcg-rinfo - display information about the given reservation(s)
•  qcg-rlist - list reservation in the system

QCG-Simple description format

Every task submitted to the system has to be described in a formal way. The current section describes the QCG-Simple format which is recommended and sufficient for most tasks. The format does not allow to describe more sophisticated scenarios like workflows, parameter sweep tasks, parallel tasks with topologies. Such scenarios are supported by the XML format called QCG-JobProfile. The QCG-Simple description format is similar to the concept used in queueing systems. The document submitted to be processed by the system is an script with QCG directives and invocation of user code. The QCG directives inform the system how to process the task (eg. about resource requirements and input/output files and directories). The QCG directives are lines starting with the #QCG string.

The detailed description of the #QCG directives The detailed description of the #QCG directives

List of #QCG directives:

•  application - name of the application
•  argument - argument of the application
•  deadline- deadline for the task execution
•  environment - environment variable
•  error - path for the standard error
•  executable - path to the executable
•  grant- name of the grant
•  host - name of the candidate host
•  input - path of the standard input file
•  memory - memory requirements
•  module- name of the required module
•  monitor - task monitoring script
•  name- human readable name of the task
•  native - parameters/requirements passed directly to the queueing system
•  nodes - requested topology of parallel application
•  not-after - execute not later then
•  not-before - do not execute before
•  note - human readable description of the task
•  notify - notify about changes of task status
•  output - location for the standard output file
•  persistent - do not remove the working directory
•  postprocess - task epilog
•  preprocess - task prolog
•  procs - number of requested slots/processes
•  properties - node properties
•  queue - selected queueu
•  reservation - reservation identifier
•  stage-in-dir - input directory
•  stage-in-file - input file
•  stage-out-dir - output directory
•  stage-out-file - output file
•  use-reservation - use the advance reservation mechanism
•  use-scratch - use the scratch space
•  walltime - execution duration
•  watch-output - monitor the output and send notifications about the progress
•  variable - description variable (replaced only in values of QCG directives)

Tasks examples

Please visit the QCG-Simple examples page for description examples.

Contact

Please report all problems and send questions to the qcg@… address.