Changes between Initial Version and Version 1 of installation_guide

Timestamp:

06/10/11 17:31:07 (13 years ago)

Author:

piontek

Comment:

--

installation_guide

@@ +1 @@
+[[PageOutline]]
+Contact: [mailto:piontek@man.poznan.pl Tomasz Piontek] [mailto:pkopta@man.poznan.pl Piotr Kopta]
+
+= QCG Broker Installation =
+
+The !QosCosGrid Resource Management System (QCG-Broker) is an open source meta-scheduling system, which allows developers to build and deploy resource management systems for large scale distributed computing infrastructures. The QCG-Broker, based on dynamic resource selection, mapping and advanced scheduling methodology, combined with feedback control architecture, deals with dynamic Grid environment and resource management challenges, e.g. load-balancing among clusters, remote job control or file staging support. Therefore, the main goal of the QCG-Broker is to manage the whole process of remote job submission to various batch queuing systems, clusters or resources. It has been designed as an independent core component for resource management processes which can take advantage of various low-level Core Services and existing technologies. Finally, the QCG-Broker can be considered as a robust system which provides abstraction of the complex grid infrastructure as well as a toolbox which helps to form and adapts to distributing computing environments.
+
+= Requirements =
+* '''Java (>= 1.5)'''\\
+{{{
+#!div style="font-size: 90%"
+For SL5.x
+{{{#!sh
+yum install java-1.6.0-sun-compat.x86_64
+}}}
+}}}
+* '''PostgreSQL (>= 7.0)''' - configured to allow TCP/IP connections from localhost (default configuration)
+{{{
+#!div style="font-size: 90%"
+For SL5.x
+{{{#!sh
+yum install postgresql-server.x86_64
+}}}
+}}}
+
+ Start the PostgreSQL service:
+{{{
+#!div style="font-size: 90%"
+For SL5.x
+{{{#!sh
+service postgresql start
+}}}
+}}}
+* '''Apache Ant''' (>= 1.6) ("Optional tasks for ant" are required. For SL5.x the ''ant-nodeps.x86_64'' package must be installed)
+{{{
+#!div style="font-size: 90%"
+For SL5.x
+{{{#!sh
+yum install ant.x86_64 ant-nodeps.x86_64
+}}}
+}}}
+* '''xml-commons-apis''' - installation of this package is not mandatory, but is recommended to avoid confusing information about not important errors.
+{{{
+#!div style="font-size: 90%"
+For SL5.x
+{{{#!sh
+yum install xml-commons.x86_64 xml-commons-apis.x86_64
+}}}
+}}}
+* '''certificates'''
+ * QCG-Broker' certificate and private key signed by one of trusted Certificate Authorities [CA],
+ * set of trusted CA certificates (must be coppied either into `/etc/grid-security/certificates` or `QCG-BROKER_USER_HOME/.globus/certificates` directory)
+
+ For the PL-Grid Project install Polish Grid and PL-Grid Simpla-CA certificates:
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+wget https://dist.eugridpma.info/distribution/igtf/current/accredited/RPMS/ca_PolishGrid-1.38-1.noarch.rpm
+wget http://software.plgrid.pl/packages/general/ca_PLGRID-SimpleCA-1.0-2.noarch.rpm
+wget https://dist.eugridpma.info/distribution/util/fetch-crl/fetch-crl-2.8.5-1.noarch.rpm
+
+su -
+rpm -i ca_PolishGrid-1.38-1.noarch.rpm 
+rpm -i ca_PLGRID-SimpleCA-1.0-2.noarch.rpm 
+
+#install certificate revocation list fetching utility
+rpm -i fetch-crl-2.8.5-1.noarch.rpm
+ 
+#get fresh CRLs now
+/usr/sbin/fetch-crl 
+ 
+#install cron job for it
+echo '#!/bin/sh' > /etc/cron.daily/fetch-crl.cron
+echo '/usr/sbin/fetch-crl' >> /etc/cron.daily/fetch-crl.cron
+chmod a+x /etc/cron.daily/fetch-crl.cron
+}}}
+}}}
+* '''range of opened ports'''
+
+  QCG-Broker needs set of opened ports to be able to receive notifications about changes of jobs and tasks statuses and to communicate with gridFTP server.The port range must be set in ~/.globus/cog.properties file.
+{{{#!sh
+mkdir -p ~/.globus
+echo "tcp.port.range=5000,5100" >> ~/.globus/cog.properties
+}}}
+
+
+= Preparing the environment =
+
+'''IMPORTANT:''' Altought it's possible to install QCG-Broker service on any account (`root`, regular user) we strongly recommend to deploy QCG-Broker service on the account designated specifically for this service (`qcg-broker` for example).
+
+We recommend to create a new user (e.g. `qcg-broker`) that will be used by the service to run in unprivileged mode
+{{{
+#!div style="font-size: 90%"
+{{{#!sh  
+mkdir -p /opt/QCG
+useradd -d /opt/QCG/qcg-broker -m -s /bin/bash qcg-broker 
+}}}
+}}}
+'''IMPORTANT:''' If you want to configure QCG-Broker to use its own list of trusted CA certificates, please do not forget to copy them into `~/.globus/certificates` directory.
+
+= Database setup =
+* Create new database user (e.g. `qcg-broker`) authenticated via password (the same password must be later used to setup QCG-Broker deployment):
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+su - postgres
+createuser -P qcg-broker
+
+Enter password for new role: 
+Enter it again: 
+Shall the new role be a superuser? (y/n) n
+Shall the new role be allowed to create databases? (y/n) y
+Shall the new role be allowed to create more new roles? (y/n) n
+CREATE ROLE
+
+exit
+}}}
+}}}
+
+* Create new database (e.g. qcg-broker) owned by the `qcg-broker` user:
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+su - qcg-broker
+createdb -U qcg-broker qcg-broker
+exit
+}}}
+}}}
+ '''Note:''' The `qcg-broker` user must be allowed to create new databases.
+
+* Depending on the local PostgreSQL configuration you may need to edit the `pg_hba.conf` file  (''host based authentication'' configuration file) to enable password authentication to the `qcg-broker` database for the `qcg-broker` user.
+
+ For SL5.x: edit the `/var/lib/pgsql/data/pg_hba.conf` file.
+
+ If the PostgreSQL installation is fully dedicated for QCG-Broker replace the default configuration with rules (lines) given below, otherwise insert them to the configuration file in a proper place:
+{{{
+#!div style="font-size: 90%"
+{{{#!default
+local   qcg-broker   qcg-broker                         md5
+host    qcg-broker   qcg-broker   127.0.0.1/32          md5
+}}}
+}}}
+
+ '''Note:''' You must reload the PostgreSQL server in order to make the changes visible e.g:
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+service postgresql reload
+}}}
+}}}
+
+= Installation =
+
+'''IMPORTANT:''' Perform the installation as the `qcg-broker` user.
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+su - qcg-broker
+}}}
+}}}
+
+* download the QCG-Broker archive from [[http://node2.qoscosgrid.man.poznan.pl/~piontek/qcg-broker/qcg-broker.tgz|qcg-broker.tgz]]
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+wget http://node2.qoscosgrid.man.poznan.pl/~piontek/qcg-broker/qcg-broker.tgz
+}}}
+}}}
+
+* unpack the archive
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+tar xzf qcg-broker.tgz
+}}}
+}}}
+
+== Compilation [Optional step] ==
+The distribution contains precompiled version of QCB-Broker, that can be deployed as it is. The compilation step is optional and can be skiped, except the situation in which some specific compiler options should be added or changed.
+
+* compile sources
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+cd qcg-broker-<VERSION>
+ant rebuildall
+}}}
+}}}
+
+== Setup ==
+* setup deployment configuration - all configuration variables are placed in the `deploy.prop` file
+ * '''deploy.dir''' - directory where QCG-Broker will be deployed. This directory must not exist, so before deployment this directory must be eventually removed by hand. Later we assume that QCG-Broker was installed into `/opt/QCG/qcg-broker/service` directory. 
+ * '''deploy.certs.dir''' - directory with QCG-Broker credentials (must contain `grmscert.pem` and `grmskey.pem`)
+ * '''deploy.cert.dn''' distinguish name of QCG-Broker credential
+ * '''deploy.db.user''' PostgreSQL user name
+ * '''deploy.db.pwd''' Password for PostgreSQL user
+ * '''deploy.db.host''' PostgreSQL database host name (`127.0.0.1` by default. If PostgreSQL service is on the same host as QCG-Broker, this property should not be changed. Otherwise some changes in PostgreSQL configuration (`pg_hba.conf` file) may be needed).
+ * '''deploy.db.name''' PostgreSQL database name
+ * '''deploy.https_port''' the port which QCG-Broker will be listening for https requests on
+ * '''deploy.rmi_port''' the port needed for internal communication between QCG-Broker components.
+ * '''deploy.shutdown_port''' the port which the tomcat will be listening for shutdown requests on
+ * '''deploy.port_range''' the range of opened ports
+
+* setup QCG-Broker database
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+cd qcg-broker-<VERSION>
+ant -f build.deploy.xml setupdb
+}}}
+}}}
+
+== Deployment ==
+Process of deployment is responsible for copping the QCG-Broker distribution to destination directory and configuring the service.
+
+* Deploy QCG-Broker service
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+ant deploy
+}}}
+}}}
+
+= Configuration =
+* Edit the `QCG-BROKER-DEPLOY_DIR/broker/etc/providers.list` file specyfying list of resource providers (instances of QCG BES/AR services controling resources and creating together virtual pool of resources)
+
+ '''providers.count'''=<n> - number of providers
+
+ For x=0 to n-1 define locations of domain providers:
+
+ '''provider.x.domain'''=<DOMAIN> - name of the administrative domain, which resources controlled by the given provider belong to,\\
+ '''provider.x.type'''=qcg - type of the provider, DO NOT CHANGE THIS VALUE,\\
+ '''provider.x.id'''=<PROVIDER ID> - identifier of the provider (must be globally unique),\\
+ '''provider.x.location'''=<LOCATION>  - address of provider service.\\
+
+ For example:
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+<pre>
+# how many providers should be used for testing
+providers.count=1
+
+provider.0.domain=PSNC
+provider.0.id=qcg_cluster
+provider.0.type=qcg
+provider.0.location=httpg://qcg.man.poznan.pl:19000
+</pre>
+}}}
+}}}
+
+* For cross-clusters job submission, edit `QCG-BROKER_DEPLOY_DIR/broker/etc/config.prop` file, and set following variables:
+ * '''broker.qoscos.proactive.pncURL''' - !ProActvie node coordinator address
+ * '''broker.qoscos.ompi.pncURL''' - OpenMPI node coordinator address
+
+ See [[installation_QCG_ProActive_Coordinator|QCG-ProActive Coordinator]], [[installation_QCG_OpenMPI_Coordinator|QCG-OpenMPI Coordinator]] for detailed information about installation of !ProActive and OpenMPI node coordinators.
+
+= Resource Providers setup =
+To allow QCG-Broker  to create and manage advance reservations on resources accessible via [[QCG_BES_AR|QCG BES/AR]] service some additional configuration steps are needed on Administrative Domain level: 
+* the QCG-Broker credential must be mapped (in the `grid-mapfile` file) to the local user that is authorized to create and manage reservations. Usually the LRMS administrator user (like `sgeadmin` in SGE or `lsfadmin` in LSF) has sufficient rights, e.g.:
+{{{
+#!div style="font-size: 90%"
+{{{#!default
+"/C=PL/O=GRID/O=PSNC/CN=qcg-broker/qcg.man.poznan.pl" sgeadmin
+}}}
+}}}
+
+= Starting the service =
+As the `qcg-broker` user type:
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+cd /opt/QCG/qcg-broker/service
+./broker/bin/start-broker.sh
+./service/bin/startup.sh
+}}}
+}}}
+
+'''IMPORTANT:''' Do not forget to set `JAVA_HOME` or `JRE_HOME` environment variables.
+= Stopping the service =
+As the `qcg-broker` user type:
+{{{
+#!div style="font-size: 90%"
+{{{#!sh
+cd /opt/QCG/qcg-broker/service
+./broker/bin/stop-broker.sh
+./service/bin/shutdown.sh
+}}}
+}}}
+= Veryfing the installation =
+To verify the installtion please use [[QCG_Broker_Client|QCG Broker Client]].
+To verify the installtion please use [[installation_QCG_Broker_Client|QCG Broker Client]].