Changes between Version 2 and Version 3 of installation_QCG_BES_AR_in_PLGrid

Show
Ignore:
Timestamp:
05/19/11 15:11:50 (9 years ago)
Author:
bartek
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • installation_QCG_BES_AR_in_PLGrid

    v2 v3  
    235235=== The Grid Mapfile  === 
    236236==== Manually created grid mapfile (for testing purpose only) ==== 
     237{{{ 
     238#!div style="font-size: 90%" 
     239{{{#!default 
    237240  #for test purpose only add mapping for your account 
    238241  echo '"MyCertDN" myaccount' >> /etc/grid-security/grid-mapfile 
     242}}} 
     243}}} 
    239244==== LDAP based grid mapfile ==== 
     245{{{ 
     246#!div style="font-size: 90%" 
     247{{{#!default 
    240248 #install grid-mapfile generator from PL-Grid repository 
    241249 yum install plggridmapfilegenerator 
     
    255263 format=^plgridX509CertificateDN, uid 
    256264 EOF 
    257   
    258265 #add the gridmapfile generator as the cron.job 
    259266 cat > /etc/cron.hourly/gridmapfile.cron << EOF 
     
    265272 #try it! 
    266273 /etc/cron.hourly/gridmapfile.cron 
    267  
     274}}} 
     275}}} 
    268276 
    269277Add appropriate rights for the smoa_comp and grms users in the Maui scheduler configuaration file: 
     278{{{ 
     279#!div style="font-size: 90%" 
     280{{{#!default 
    270281  vim /var/spool/maui/maui.cfg 
    271282  # primary admin must be first in list 
     
    273284  ADMIN2                grms 
    274285  ADMIN3                smoa_comp 
    275  
     286}}} 
     287}}} 
    276288Copy the service certificate and key into the <code>/opt/plgrid/qcg/smoa/etc/certs/</code>. Remember to set appropriate rights to the key file. 
     289{{{ 
     290#!div style="font-size: 90%" 
     291{{{#!default 
    277292  cp /etc/grid-security/hostcert.pem /opt/plgrid/qcg/smoa/etc/certs/smoacert.pem 
    278293  cp /etc/grid-security/hostkey.pem /opt/plgrid/qcg/smoa/etc/certs/smoakey.pem 
     
    280295  chown smoa_comp /opt/plgrid/qcg/smoa/etc/certs/smoakey.pem  
    281296  chmod 0600 /opt/plgrid/qcg/smoa/etc/certs/smoakey.pem 
    282  
     297}}} 
     298}}} 
    283299== DRMAA library == 
    284 * DRMAA library must be compiled from SRC RPM:  
     300DRMAA library must be compiled from SRC RPM:  
     301{{{ 
     302#!div style="font-size: 90%" 
     303{{{#!default 
    285304  wget http://fury.man.poznan.pl/qcg-packages/sl/SRPMS/pbs-drmaa-1.0.6-2.src.rpm 
    286305  rpmbuild  --rebuild pbs-drmaa-1.0.6-2.src.rpm 
    287306  cd /usr/src/redhat/RPMS/x86_64/ 
    288307  rpm -i pbs-drmaa-1.0.6-2.x86_64.rpm  
    289  
    290 * however if you are using it for the first time then you should try to compile it with enabled logging: 
     308}}} 
     309}}} 
     310however if you are using it for the first time then you should try to compile it with enabled logging: 
     311{{{ 
     312#!div style="font-size: 90%" 
     313{{{#!default 
    291314  wget http://fury.man.poznan.pl/qcg-packages/sl/SRPMS/pbs-drmaa-1.0.6-2.src.rpm 
    292315  rpmbuild  --define 'configure_options --enable-debug' --rebuild pbs-drmaa-1.0.6-2.src.rpm 
    293316  cd /usr/src/redhat/RPMS/x86_64/ 
    294317  rpm -i pbs-drmaa-1.0.6-2.x86_64.rpm 
    295  
     318}}} 
     319}}} 
    296320After installation  you need '''either''': 
    297 * configure the DRMAA library to use Torque logs ('''RECOMMENDED'''). Sample configuration file of the DRMAA library (<code>/opt/plgrid/qcg/smoa/etc/pbs_drmaa.conf</code>): 
     321* configure the DRMAA library to use Torque logs ('''RECOMMENDED'''). Sample configuration file of the DRMAA library (`/opt/plgrid/qcg/smoa/etc/pbs_drmaa.conf`): 
     322{{{ 
     323#!div style="font-size: 90%" 
     324{{{#!default 
    298325  # pbs_drmaa.conf - Sample pbs_drmaa configuration file. 
    299326   
     
    303330     
    304331  cache_job_state: 600, 
    305 {{Note}} Remember to mount server log directory as described in the eariler [[Smoa_Computing_on_separate_machine|note]]. 
     332}}} 
     333}}} 
     334'''Note:''' Remember to mount server log directory as described in the eariler [[Smoa_Computing_on_separate_machine|note]]. 
    306335 
    307336'''or''' 
    308 * configure Torque to keep information about completed jobs (e.g.: by setting: qmgr -c 'set server keep_completed = 300'). 
    309    
    310 It is possible to limit users to submit job to predefined queue by setting default job category (in the <code>/opt/plgrid/qcg/smoa/etc/pbs_drmaa.conf</code> file): 
    311  
     337* configure Torque to keep information about completed jobs (e.g.: by setting: `qmgr -c 'set server keep_completed = 300'`). 
     338   
     339It is possible to limit users to submit job to predefined queue by setting default job category (in the `/opt/plgrid/qcg/smoa/etc/pbs_drmaa.conf` file): 
     340{{{ 
     341#!div style="font-size: 90%" 
     342{{{#!default 
    312343  job_categories: { 
    313344        default: "-q plgrid", 
    314345  }, 
     346}}} 
     347}}} 
    315348 
    316349== Restricting advance reservation == 
    317 In some deployments enabling advance reservation for the whole cluster is not desirable. In such cases one can limit advance reservation to particular partition by editing <code>/opt/plgrid/qcg/smoa/lib/smoa-comp/modules/python/reservation_maui.py</code> file and changing the following line: 
     350In some deployments enabling advance reservation for the whole cluster is not desirable. In such cases one can limit advance reservation to particular partition by editing `/opt/plgrid/qcg/smoa/lib/smoa-comp/modules/python/reservation_maui.py` file and changing the following line: 
     351{{{ 
     352#!div style="font-size: 90%" 
     353{{{#!default 
    318354  cmd = "setres -x BYNAME -r PROCS=1" 
     355}}} 
     356}}} 
    319357to 
     358{{{ 
     359#!div style="font-size: 90%" 
     360{{{#!default 
    320361  cmd = "setres -x BYNAME -r PROCS=1 -p wp4" 
     362}}} 
     363}}} 
    321364 
    322365= Service configuration = 
    323 Edit the preinstalled service configuration file (<code>/opt/plgrid/qcg/smoa/etc/smoa-compd.xml</code>): 
    324  
     366Edit the preinstalled service configuration file (`/opt/plgrid/qcg/smoa/etc/smoa-compd.xml`): 
     367{{{ 
     368#!div style="font-size: 90%" 
     369{{{#!xml 
    325370  <?xml version="1.0" encoding="UTF-8"?> 
    326371  <sm:SMOACore 
     
    399444        </Configuration> 
    400445  </sm:SMOACore> 
    401   <!-- vim: set ts=2 sw=2: --> 
     446}}} 
     447}}} 
    402448 
    403449In most cases it should be enough to change only following elements: 
    404 ; ''Transport/Module/Host'' :  the hostname of the machine where the service is deployed  
    405 ; ''Transport/Module/Authentication/Module/X509CertFile'' and  ''Transport/Module/Authentication/Module/X509KeyFile'' : - the service private key and X.509 certificate (consult the [http://www.globus.org/toolkit/docs/4.0/security/prewsaa/rn01re02.html Globus User Gide] on how to generate service certificate request or use the host certificate/key pair). Make sure that the key and certificate is owned by the <code>smoa_comp</code> user and the private key is not password protected (generating certificate with the <code>-service</code> option implies this). If you installed cert and key file in the recommended location you do not need to edit these fields. 
    406 ; ''Module[type="smc:notification_wsn"]/Module/ServiceURL'' : the URL of the [[SMOA_Notification_in_PL-Grid|Smoa Notification Service]] (You can do it later, i.e. after installing the Smoa Notification service) 
    407 ;  Module[type="submission_drmaa"]/@path : path to the DRMAA library (the <code>libdrmaa.so</code>). Also, if you installed the DRMAA library using provided SRC RPM you do not need to change this path. 
    408 ;  ''Database/Password'' : the <code>smoa_comp</code> database password 
    409 ;  ''FactoryAttributes/CommonName'' : a common name of the cluster (e.g. reef.man.poznan.pl). You can use any name that is unique among all systems (e.g. cluster name + domain name of your institution) 
    410 ;  ''FactoryAttributes/LongDescription'' : a human readable description of the cluster 
     450 `Transport/Module/Host` :: 
     451   the hostname of the machine where the service is deployed  
     452 `Transport/Module/Authentication/Module/X509CertFile`  and  `Transport/Module/Authentication/Module/X509KeyFile` ::  
     453  the service private key and X.509 certificate (consult the [[http://www.globus.org/toolkit/docs/4.0/security/prewsaa/rn01re02.html|Globus User Gide]] on how to generate service certificate request or use the host certificate/key pair). Make sure that the key and certificate is owned by the `smoa_comp` user and the private key is not password protected (generating certificate with the `-service` option implies this). If you installed cert and key file in the recommended location you do not need to edit these fields. 
     454 `Module[type="smc:notification_wsn"]/Module/ServiceURL` ::  
     455  the URL of the [[SMOA_Notification_in_PL-Grid|Smoa Notification Service]] (You can do it later, i.e. after installing the Smoa Notification service) 
     456 `Module[type="submission_drmaa"]/@path` :: 
     457  path to the DRMAA library (the `libdrmaa.so`). Also, if you installed the DRMAA library using provided SRC RPM you do not need to change this path. 
     458 `Database/Password` ::  
     459  the `smoa_comp` database password 
     460 `FactoryAttributes/CommonName` ::  
     461  a common name of the cluster (e.g. reef.man.poznan.pl). You can use any name that is unique among all systems (e.g. cluster name + domain name of your institution) 
     462 `FactoryAttributes/LongDescription` ::  
     463  a human readable description of the cluster 
    411464 
    412465== Configuring BAT accounting module == 
    413 In order to report resource usage to the central PL-Grid accounting service you must enable the <code>bat_updater</code> module. You can do this by including the following snippet in the aforementioned configuration file (<code>/opt/plgrid/qcg/smoa/etc/smoa-comp.xml</code>). Please put the following snippet just before the <code>Database</code> section: 
     466In order to report resource usage to the central PL-Grid accounting service you must enable the `bat_updater` module. You can do this by including the following snippet in the aforementioned configuration file (`/opt/plgrid/qcg/smoa/etc/smoa-comp.xml`). Please put the following snippet just before the `Database` section: 
     467{{{ 
     468#!div style="font-size: 90%" 
     469{{{#!xml 
    414470  <sm:Module xsi:type="bat_updater"> 
    415471        <BATServiceURL>tcp://acct.grid.cyf-kr.edu.pl:61616</BATServiceURL> 
     
    417473        <QueueName>test-jobs</QueueName> 
    418474  </sm:Module> 
    419  
     475}}} 
     476}}} 
    420477where: 
    421 ;BATServiceURL : URL of the BAT accounting service 
    422 ;SiteName : local site name as reported to the BAT service 
    423 ;QueueName : queue name to which report usage data 
     478* BATServiceURL : URL of the BAT accounting service 
     479* !SiteName : local site name as reported to the BAT service 
     480* !QueueName : queue name to which report usage data 
    424481 
    425482= Note on the security model = 
    426483The Smoa Computing can be configured with various authentication and authorization modules. However in the typical deployment we assume that the Smoa Computing is configured as in the above example, i.e.: 
    427484* authentication is provided on basics of ''httpg'' protocol 
    428 * authorization is based on the local <code>grid-mapfile</code> mapfile (see [[GridFTP#Users_configuration|  Users configuration]]). 
     485* authorization is based on the local `grid-mapfile` mapfile (see [[GridFTP#Users_configuration|Users configuration]]). 
    429486 
    430487=Starting the service= 
    431488As root type: 
    432   
     489{{{ 
     490#!div style="font-size: 90%" 
     491{{{#!sh 
    433492 # /etc/init.d/smoa-compd start 
     493}}} 
     494}}} 
    434495 
    435496The service logs can be found in: 
     497{{{ 
     498#!div style="font-size: 90%" 
     499{{{#!sh 
    436500  /opt/plgrid/var/log/smoa-comp/smoa-comp.log 
     501}}} 
     502}}} 
    437503 
    438504The service assumes that the following commands are in the standard search path: 
    439 * pbsnodes 
    440 * showres 
    441 * setres 
    442 * releaseres 
    443 * checknode 
    444 If any of the above commands is not installed in a standard location (e.g. <code>/usr/bin</code>) you may need to edit the <code>/opt/plgrid/qcg/smoa/etc/sysconfig/smoa-compd</code> file and set the PATH variable appropriately, e.g.: 
     505* `pbsnodes` 
     506* `showres` 
     507* `setres` 
     508* `releaseres` 
     509* `checknode` 
     510If any of the above commands is not installed in a standard location (e.g. `/usr/bin`) you may need to edit the `/opt/plgrid/qcg/smoa/etc/sysconfig/smoa-compd` file and set the `PATH` variable appropriately, e.g.: 
     511{{{ 
     512#!div style="font-size: 90%" 
     513{{{#!sh 
    445514  # INIT_WAIT=5 
    446515  # 
     
    448517   
    449518  export PATH=$PATH:/opt/maui/bin 
     519}}} 
     520}}} 
    450521 
    451522If you compiled DRMAA with logging switched on you can set there also DRMAA logging level: 
     523{{{ 
     524#!div style="font-size: 90%" 
     525{{{#!sh 
    452526  # INIT_WAIT=5 
    453527  # 
     
    455529 
    456530  export DRMAA_LOG_LEVEL=INFO 
    457  
    458 =Stopping the service= 
     531}}} 
     532}}} 
     533 
     534= Stopping the service = 
    459535The service can be stopped using the following command: 
     536{{{ 
     537#!div style="font-size: 90%" 
     538{{{#!sh 
    460539  # /etc/init.d/smoa-compd stop 
    461  
    462 =Verifying the installation= 
    463  
    464 *  For convenience you can add the <code>/opt/plgrid/qcg/smoa/bin</code> and <code>/opt/plgrid/qcg/smoa-dep/globus/bin/</code> to your <code>PATH</code> variable. 
    465 *  Edit the Smoa Computing client configuration file (<code>/opt/plgrid/qcg/smoa/etc/smoa-comp.xml</code>): 
    466 **  set the ''Host'' and ''Port''  to reflects the changes in the service configuration file (<code>smoa-compd.xml</code>). 
    467  
     540}}} 
     541}}} 
     542 
     543= Verifying the installation = 
     544 
     545*  For convenience you can add the `/opt/plgrid/qcg/smoa/bin` and `/opt/plgrid/qcg/smoa-dep/globus/bin/` to your `PATH` variable. 
     546*  Edit the Smoa Computing client configuration file (`/opt/plgrid/qcg/smoa/etc/smoa-comp.xml`): 
     547 *  set the `Host` and `Port` to reflects the changes in the service configuration file (`smoa-compd.xml`). 
     548{{{ 
     549#!div style="font-size: 90%" 
     550{{{#!sh 
    468551 <?xml version="1.0" encoding="UTF-8"?> 
    469552 <sm:SMOACore 
     
    492575        </Configuration> 
    493576 </sm:SMOACore> 
    494  
     577}}} 
     578}}} 
    495579* Initialize your credentials: 
    496  
     580{{{ 
     581#!div style="font-size: 90%" 
     582{{{#!sh 
    497583 $ grid-proxy-init  
    498584 Your identity: /O=Grid/OU=QosCosGrid/OU=PSNC/CN=Mariusz Mamonski 
     
    500586 Creating proxy .................................................................. Done 
    501587 Your proxy is valid until: Wed Sep 16 05:01:02 2009 
    502    
     588}}} 
     589}}} 
    503590* Query the SMOA Computing service: 
     591{{{ 
     592#!div style="font-size: 90%" 
     593{{{#!sh 
    504594  $ smoa-comp -G | xmllint --format - # the xmllint is used only to present the result in more pleasant way 
    505595   
     
    523613    <smcf:NotificationProviderURL xmlns:smcf="http://schemas.smoa-project.com/comp/2009/01/factory">http://localhost:2211/</smcf:NotificationProviderURL> 
    524614 </bes-factory:FactoryResourceAttributesDocument> 
    525  
     615}}} 
     616}}} 
    526617* Submit a sample job: 
     618{{{ 
     619#!div style="font-size: 90%" 
     620{{{#!sh 
    527621  $ smoa-comp -c -J /opt/plgrid/qcg/smoa/share/smoa-comp/doc/examples/jsdl/sleep.xml 
    528622  Activity Id: ccb6b04a-887b-4027-633f-412375559d73 
    529  
     623}}} 
     624}}} 
    530625* Query it status: 
     626{{{ 
     627#!div style="font-size: 90%" 
     628{{{#!sh 
    531629  $ smoa-comp -s -a ccb6b04a-887b-4027-633f-412375559d73 
    532630  status = Executing 
     
    536634  status = Finished 
    537635  exit status = 0 
    538  
     636}}} 
     637}}} 
    539638* Create an advance reservation: 
    540639:* copy the provided sample reservation description file (expressed in ARDL - Advance Reservation Description Language)