Upgrading DPM Upgrading DPM

About 

This tutorial aims to detail the steps needed to perform an upgrade of Disk Pool Manager from the version distributed with gLite to the EMI 2 version. 

Introduction

For site administrators running older versions of grid middleware, upgrading to the new EMI versions can be quite a time consuming process. Because of the changes between the EMI versions, nodes are often completely reinstalled when upgrading. Some packages, however, can be updated without having to reinstall the operating system or restart the machine, allowing upgrades to occur with a minimal interruption of service.
 
In this tutorial we show how to migrate an existing working production instance of DPM from gLite 3.2 version to the EMI 2 one. Both gLite 3.2 and EMI 2 support SL5 64 bit, and so there is no need to reinstall the entire machine, and the entire process can be completed in a matter of minutes, with the services automatically restarted by YAIM on reconfiguration. It's wise however to backup the backend MySQL DB. Also, the tutorial makes use of the GILDA VO, but the same steps work for any VO. 
 
Please note that it is good practice to review the EMI 2 Generic Installation and Configuration Guide before embarking on any installation or upgrade, in particular if you are working with a production system. While the required steps for performing an upgrade of DPM are listed in this tutorial, it should not be considered a definitive guide, and more information may be available at the above page.

Preliminary checks

In order to perform an upgrade you should first check that the gLite DPM service is installed and working on your machine, and check its version.
 
check that the service is published to the BDII by running lcg-infosites, for example:
 
$ lcg-infosites --vo gilda se
Avail Space(Kb) Used Space(Kb)  Type    SEs
----------------------------------------------------------
428613480       99825233        n.a     vm10.ct.infn.it
…..
428613620       99825093        n.a     aliserv6.ct.infn.it
 
In our case the host in question is vm10.ct.infn.it, and we can see that it is being correctly published. Next we should test that we can actually register a file. Create a file called test.txt and then run the following command:
 
$ lcg-cr -d vm10.ct.infn.it file:~/test.txt
guid:5ec22ba1-37d8-4a78-b321-4a7c3234e686
 
We can check the SURL of the newly created file as follows, to verify that it has been created on our DPM node:
 
$ lcg-lr guid:5ec22ba1-37d8-4a78-b321-4a7c3234e686
srm://vm10.ct.infn.it/dpm/ct.infn.it/home/gilda/generated/2012-09-18/file18205d44-b6a3-41b8-bdcf-eb902f83c77b
 
Before upgrading we should be sure what version of DPM we are currently running. The procedure here has been tested for upgrading from DPM 1.8.1 and above, to the current EMI 2 version of DPM, 1.8.4, but it should also work for upgrading from older versions.
 
$ rpm -qa | grep DPM-server
DPM-server-mysql-1.8.2-3sec.sl5

Update Repositories

Next you should remove any old repositories and install the new EMI repository so that you can get the latest packages:
 
The DAG repository is a repository of add-on RPM packages for Red Hat/Fedora based systems. Packages in this repository can conflict with the versions required by EMI products, so you should disable this repository if it is enabled on your system. To remove the DAG repository you should set enabled=0 in the file /etc/yum.repos.d/dag.repo.
 
You should do the same for any glite repositories, or other old or non-standard repositories. For example, in our case we had a machine in the Italian National Grid which had an Italian Grid repository, the GILDA repository, as well as the gLite DPM and UI repositories. Check the directory /etc/yum.repos.d/ to find which repositories are enabled on your machine.
 
Next you must install both the EMI and EPEL repositories. The EPEL repository contains many of the external dependencies of the EMI products:
 
$ yum install  yum-protectbase yum-priorities
$ rpm -Uvh http://download.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
$ rpm -Uvh http://emisoft.web.cern.ch/emisoft/dist/EMI/2/sl5/x86_64/base/emi-release-2.0.0-1.sl5.noarch.rpm

Backup siteinfo files

When you upgrade YAIM it will download a new set of example configuration files, so it is good practice to always back-up your customised site-info.def and other configuration files before performing any upgrades, to avoid losing your configuration. You can use the same files with only minor edits to configure the new version of DPM. Your configuration files may be in the /opt/glite/yaim/examples/siteinfo/ directory if you did not create a custom directory for them.
 
Copy your customised versions of the following files to a safe backup location:
 
site-info.def
services/*
vo.d/*
users.conf
groups.conf
 
For example:
 
$ mkdir /root/yaimconfig
$ cd /opt/glite/yaim/examples/siteinfo
$ cp -r site-info.def services vo.d users.conf groups.conf /root/yaimconfig/
 
Once you have backed up your configuration you will need to make some small adjustments. Edit the site-info.def file and change the USERS_CONF and GROUPS_CONF variables to point to the new location to which you have backed up these files.

Remove conflicting packages

Some of the gLite 3.2 packages already installed on your system might conflict with the packages which you are installing, or their dependencies. You will need to remove any potentially conflicting packages before you can install the new version. In the case of DPM, you must remove the following packages:
  • vdt_globus_essentials
  • vdt_globus_data_server 
  • lcg-service-proxy
  • glite-yaim-dpm
  • ig-yaim
It is a good idea to remove these using the yum erase command as this will also remove any conflicting dependencies:
 
$ yum erase vdt_globus_essentials vdt_globus_data_server lcg-service-proxy glite-yaim-dpm ig-yaim

Install the new version 

You are now ready to install the new middleware. To be sure that your system is up to date, you should first clear the yum cache and run a yum update to get the latest versions of all installed packages:
 
$ yum clean all && yum update
 
Then install the required EMI package or metapackage, in this case emi-dpm_mysql:
 
$ yum install -y emi-dpm_mysql

Package-specific actions

Some EMI components have changed quite significantly between the older gLite versions and the EMI 2 versions. When upgrading any package it is likely that there will be some workarounds and fixes which will have to be performed, and DPM is no exception.
 
First you should perform a mysql upgrade. For this you will need the mysql root password, you can find this in MYSQL_PASSWORD variable of the site-info.def file. Once you have the password, simply run the following command:
 
$ mysql_upgrade -u root -p
 
In the siteinfo/services directory you will need to rename two files:
 
$ mv glite-se_dpm_mysql emi_dpm_mysql
$ mv glite-se_dpm_disk  emi_dpm_disk 
 
If it is not already present you must define the variable SITE_SUPPORT_EMAIL in the site-info.def file.
 
It is not strictly related to DPM, but with the EMI distribution it is necessary to change the ownership on the BDII log:
 
$ chown ldap.ldap /var/log/bdii/bdii-update.log
 
Finally you should change the GLOBUS Location by editing the file /etc/init.d/dpm-gsiftp and setting GLOBUS_LOCATION=/usr.
 
Note that when installing any EMI component it is helpful to review the release notes on the EMI webpage products list. Here you will find information on any known issues with the current released version of each package, along with workarounds and fixes for these. If you encounter problems when installing or testing any of the components please check the release notes to see if your problem is a known issue.

Configure with YAIM

You should now be ready to perform the YAIM configuration by using the following command:
 
$ /opt/glite/yaim/bin/yaim -c -s /root/yaimconfig/site-info.def -n emi_dpm_mysql
 
To check if this has worked you should try to view the file that you created previously, and try creating and deleting files:
 
$ lcg-lr guid:5ec22ba1-37d8-4a78-b321-4a7c3234e686
srm://vm10.ct.infn.it/dpm/ct.infn.it/home/gilda/generated/2012-09-18/file18205d44-b6a3-41b8-bdcf-eb902f83c77b
$ lcg-del -a guid:5ec22ba1-37d8-4a78-b321-4a7c3234e686
$ lcg-cr -d vm10.ct.infn.it file:/home/giorgio/hostname.jdl 
guid:9f763901-e197-40b1-b479-b66970c6cf63
$ lcg-del -a guid:9f763901-e197-40b1-b479-b66970c6cf63