Introduction

The QCG-Broker is a successor of the Grid Resource Management System (GRMS) Project. QCG-Broker was designed to be an open-source meta-scheduling framework that allows developers to build and easily deploy resource management systems to control large-scale distributed computing infrastructures running queuing or batch systems locally. Based on dynamic resource selection, advance reservation and various scheduling methodologies, combined with feedback control architecture, QCG-Broker deals efficiently with various meta-scheduling challenges, e.g., co-allocation, load-balancing among clusters, remote job control, file staging support or job migration. The main goal of QCG-Broker was to manage the whole process of remote job submission and advance reservation to various batch queuing systems and subsequently to underlying clusters and computational resources. It has been designed as an independent core component for resource management processes which can take advantage of various low-level core and grid services and existing technologies, such as QCG-Computing or QCG-Notification, as well as various grid middleware services such as gLite, Globus or Unicore. Addressing various demanding computational needs of large-scale complex simulations, which in many cases can exceed capabilities of a single cluster, QCG-Broker can flexibly distribute and control applications onto many computing clusters or supercomputers on behalf of end users. Moreover, owing to some built-in metascheduling procedures it can optimize and run efficiently a wide range of applications while at the same time increasing the overall throughput of computing e-infrastructures. Advance reservation mechanisms are used to create, synchronize and simultaneously manage the co-allocation of computing resources located at different Administrative Domains. The XML-based job definition language Job Profile makes it relatively easy to specify the requirements of large-scale parallel applications together with the complex parallel communication topologies. Consequently, application developers and end users are able to run their experiments in parallel over multiple clusters as well to perform various benchmark-based experiments as alternative topologies are taken into account during meta-scheduling processes in QCG-Broker.

Main Features

  • submission and control of many kinds of computational experiments including:
    • workflow experiments defined as set of tasks with precedence constraints in the form of task's states relationships,
    • multi-mimensional parameter sweep experiments,
    • parallel and hybrid applications,
  • support for Quality of Service (e.g. start time) based on advance reservation mechanisms,
  • support for co-allocation of resources and cross-cluster executions of applications
  • shorter waiting times and improved resource utilization by hierarchical grid- and local-level job scheduling,
  • management of input and output files in distributed computing clusters,
  • secure communication channels using transport level (SSL/TLS, X.509) and message level (SAML2.0) mechanisms,
  • delivered together with command-line, graphical, web-based tools for end users and administrators,
  • fast and reliable installation procedures.

QCG-Broker Service

Installation

For the detailed information describing the installation procedure of QCG-Broker Service please visit:

Release Notes

QCG-Broker Client

QCG-Broker client is fully functional java based commandline client for QCB-Broker service.

Installation

For the detailed information describing the installation procedure of QCG-Broker Client please visit:

Release Notes

User Guide

For the information how to use the QCG-Broker client and how to describe computation experiments using the QCG Job Profile please visit:

QCG-Broker Nagios Probe (for admins only)

For the information how to install QCG-Broker Probe for  Nagios monitoring system please visit:

QCG-Broker BDII Provider (for admins only)

For the information how to integrate QCG-Broker with BDII Information System please visit: