[[PageOutline]] = !QosCosGrid = The !QosCosGrid (QCG) middleware is an integrated e-infrastructure offering advanced job and resource management capabilities to deliver to end-users supercomputer-like performance and structure. By connecting many computing clusters together, !QosCosGrid offers easy-to-use mapping, execution and monitoring capabilities for variety of applications, such as parameter sweep, workflows, MPI or hybrid MPI-OpenMP. Thanks to !QosCosGrid, large-scale and complex computing models written in Fortran, C, C++ or Java can be automatically distributed over a network of computing resources with guaranteed Quality of Service. Consequently, applications can be run at given periods of time, their execution time and waiting times can be reduced, and thus bigger problem instances can be considered. == News == '''!QosCosGrid in Mapper''' ''April 2011'' It was decided to use the !QosCosGrid middleware to multi-scale computations in the Mapper project. Installations of the !QosCosGrid middleware at production sites in Europe, including EGI and PRACE machines, are currently beeing discussed. '''QCG release 2.0''' ''16 April 2011'' We are happy to introduce a next relase of !QosCosGrid. This is the first release after finishing the !QosCosGrid project and takes a number 2.0. '''!QosCosGrid middleware deployed in PL-Grid''' ''March - April 2011'' !QosCosGrid was successfully deployed on the PL-Grid infrastructure in Poland. Currently the QCG middleware is available in Poznan Supercomputing and Networking Center as well as in AGH Cyfronet in Krakow. Further deployment are planned. '''!QosCosGrid project page moved''' ''March 2010'' After the end of !QosCosGrid project, the old www.qoscosgrid.org page was moved to the [[http://biomed.science.ulster.ac.uk/archive/www.qoscosgrid.eu/|archive]]. The new page is strictly related to the !QosCosGrid middleware and further !QosCosGrid sollutions. == QCG Middleware == === Main features === * the most efficient remote access to computational resources in a single cluster or many clusters in Poland and Europe, * automatic steering of various types of complex computing experiments ranging from multi-parameter sweep studies to cross-cluster executions of parallel applications, * fully transparent integration with parallel programming and execution environments like OpenMPI and !ProActive located on many computing clusters, * support for Quality of Service (e.g. start time) based on advance reservation mechanisms, * shorter waiting times and improved resource utilization by hierarchical grid- and local-level job scheduling, * management of input and output files in distributed computing clusters, * efficient integration between services and queuing systems ensuring high performance and reliability of the overall system, * extensible, open and standard based architecture supporting OGF DRMAA, JSDL, BES and HPC Profile with pluggable modules, * secure communication channels using transport level (SSL/TLS, X.509) and message level (SAML2.0) mechanisms, * delivered together with command-line, graphical, web-based or even mobile phone tools for end users and administrators, * fast and reliable installation procedures. === Architecture === The following diagram presents a general architecture of the !QosCosGrid middleware. Information about individual components is available by clicking on the corresponding objects. {{{ #!html Cross-cluster communication QCG Science Gateways QCG Data Movement QCG Data Movement QCG Broker QCG BES/AR QCG BES/AR QCG-Notification QCG-Notification }}} In a nutshell, the !QosCosGrid middleware consists of two logical levels: grid domain and administrative domain. Grid-level services control, schedule and generally supervise the execution of end-users applications, which are spread between independent administrative domains. The administrative domain represents a single resource provider (e.g. HPC or datacenter) participating in a certain Grid or Cloud environment by sharing its computational resources, e.g. computing clusters, with both local and external end-users. The logical separation of administrative domains corresponds with the fact that they are possessed by different institutions or resource owners. Each institution contributes its resources for the benefit of the entire Grid or Cloud, while controlling its own administrative domain and own resource allocation/sharing policies. The key component of every administrative domain in !QosCosGrid is the [[QCG-Computing|QCG-Computing]], which gives the remote access to queuing systems resources. QCG-Computing supports advance reservations, parallel execution environments - OpenMPI, !ProActive and MUSCLE with coordinators responsible for synchronization of cross-cluster executions (see [[QCG_libraries_and_cross-cluster_communication|Cross-cluster communication]]) and [[QCG_DataMovement|QCG Data Movement services]] for managing input and output data. The another relevant service at the administrative domain is in charge of notification mechanism and it is called [[QCG-Notification|QCG-Notification]]. These services are tightly integrated and connected to the Grid-level services. The critical service on that level is [[QCG-Broker|QCG-Broker]], which is a meta-scheduling framework controlling executions of applicaitons on the top of queuing systems via QCG-Computing services. == End-user Information == The demanding nature of scientific simulations requires a new e-Infrastructure that is able to simultaneously manage heterogeneous resources, such as computing resources, storage and network to guarantee the level of Quality of Service (QoS) required by end-users for their applications, especially a large number of legacy applications designed to run in parallel. Addressing the need of end-users to run large scale complex simulations we have built a system capable to bring supercomputer-like performance for advanced applications, including sophisticated parameter sweep experiments, workflows, and more importantly cross-cluster parallel computations. For high-level end-users we provide a QCG Science Gateways, the components offering an easy access to concrete applications. If you are interested in using QCG as a user see [[user_information|End-user Information]]. == Developer & Admin Information == !QosCosGrid provides a consistent environment for creating and deploying Grid applications. It is also relatively easy to install, configure and maintain. Developers of new applications and administrators should look at the [[developer_and_admin_information|Developer & Admin Information]] section for more details. == Community == The !QosCosGrid community is still rising. See the [[community|Community]] section and get involved! == Installation == For the !QosCosGrid packages and installation istructions go to the [[installation|Installation]] section. == Interoperability & standards supported == The !QosCosGrid implementation is based on the open, widely accepted standards. In general, !QosCosGrid supports OGF DRMAA, JSDL, BES, HPC Profile and OASIS WS-Notification. == Contacts == Application Department of [[http://www.man.poznan.pl|Poznan Supercomputing and Networking Center]] * krzysztof.kurowski(at)man.poznan.pl * piontek(at)man.poznan.pl * mamonski(at)man.poznan.pl * bbosak(at)man.poznan.pl