Copyright Notice
All Rights Reserved Legend
US Government Users Restricted Rights Notice
UPGRADING BLUE GENE/L FROM V1R3M3 to V1R3M4
(ReadMe_BGL.txt version 3)
CONTENTS
BEFORE YOU BEGIN
UPGRADE INSTRUCTIONS
BACKING OFF THE NEW RELEASE
DB2 VERSION 9.1 SUPPORTED
SPECIAL INSTRUCTIONS FOR GPFS
COMPATIBILITY OF OTHER IBM PRODUCTS WITH BLUE GENE/L V1R3M3
V1R3M4 MEMO TO USERS
UPGRADING FROM V1R3M3
BUILDING THE BLRTS TOOLCHAIN
ADDITIONAL ENHANCEMENTS AND FIXES
MISCELLANEOUS ACKNOWLEDGEMENTS
CHECKSUMS
BEFORE YOU BEGIN
A post-install script will build the BLRTS toolchain if directory
/bgl/downloads exists on the Service Node and it contains these 4 GNU
source tar files.
binutils-2.16.1.tar.gz
gcc-3.4.3.tar.bz2
gdb-6.4.tar.gz
glibc-2.3.6.tar.gz
Check to see if the /bgl/downloads directory exists, and if the four
.tar files listed above are contained in it. If the directory doesn.t
exist, and/or the files aren.t in it, go to the BUILDING THE BLRTS
TOOLCHAIN section later in this document, for the action to take.
UPGRADE INSTRUCTIONS
These instructions are for upgrading from V1R3M3 to V1R3M4 only.
Note that steps 1 through 8 can be done ahead of time, because you
don't need to stop the control system until step 9.
1. mkdir /bgl/rpms/V1R3M4_300_2008-080728 to store the new RPMs on your
Service Node.
Download RPMs to /bgl/rpms/V1R3M4_300_2008-080728
From
https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=BGL-BLUEGENE
You'll need two of the Offerings:
Blue Gene/L Programs
GNU Toolchain Patches for CNK
2. Install the RPMs and automatically build the BLRTS toolchain. See
CHECKSUMS section at end of readme for complete list of RPMs.
NOTE: The toolchain build will take 30-45 minutes to complete.
cd /bgl/rpms/V1R3M4_300_2008-080728
rpm -ivh *rpm
If you get message
"RPM has installed but automatic building of the blrts
toolchain not successful",
refer to the BUILDING THE BLRTS TOOLCHAIN section later in this
document, for the action to take. Then continue as follows.
When the toolchain build is complete, check the last line of
/bgl/downloads/buildlog.blrts. If it does not contain the word
"Error", your BLRTS toolchain build was successful.
3. Update db.properties
diff /bgl/BlueLight/ppcfloor/bglsys/bin/db.properties /bgl/BlueLight/V1R3M4_300_2008-080728/ppc/bglsys/bin/db.properties
If you've changed any of the values in
/bgl/BlueLight/ppcfloor/bglsys/bin/db.properties (for example, the DB2
userid or password), update
/bgl/BlueLight/V1R3M4_300_2008-080728/ppc/bglsys/bin/db.properties with
those same changes.
4. Update discovery directory
If you modified any files in
/bgl/BlueLight/V1R3M3_420_2007-071023/ppc/bglsys/discovery/
other than runPopIpPool and ServiceNetwork.cfg you'll need to make
similar modifications to the same files in
/bgl/BlueLight/V1R3M4_300_2008-080728/ppc/bglsys/discovery/.
5. Customize control system scripts
If you've customized any of these files in
/bgl/BlueLight/V1R3M3_420_2007-071023/ppc/bglsys/bin:
bglmaster.init
startciodb
startidoproxy
startmmcs
startmon
startperfmon
you will probably want to make the same customizations in
/bgl/BlueLight/V1R3M4_300_2008-080728/ppc/bglsys/bin.
6. Customize Service Action Tool scripts
If you have customized the file
/bgl/BlueLight/V1R3M3_420_2007-071023/ppc/bglsys/tools/serviceaction/sa.cfg
you will probably want to make the same customizations in
/bgl/BlueLight/V1R3M4_300_2008-080728/ppc/bglsys/tools/serviceaction/sa.cfg
7. Configure Blue Gene Navigator
If you want to allow users access to a subset of the Navigator pages,
run these commands. For details on end-user access to the Navigator,
see "V1R3M2 Blue Gene/L: System Administration Redbook".
cd /bgl/BlueLight/V1R3M4_300_2008-080728/ppc/web
cp web-withenduser.xml BlueGeneNavigator/WEB-INF/web.xml
8. Stop the control system jobs and tomcat running on the service node.
(These are sample commands. You are probably not running the discovery
and system controller jobs.)
cd /discovery
./SystemController stop
./Discovery0 stop // This is for the first row of BGL racks.
./Discovery1 stop // This is for the second row of BGL racks.
…
./PostDiscovery stop
/etc/init.d/tomcat stop
/bgl/BlueLight/ppcfloor/bglsys/bin/bglmaster stop
9. Update symlinks to DB2 files
Run this command to remove DB2 Version 8.1 DB2 symlinks for tomcat.
cd /bgl/BlueLight/ppcfloor/web
./tomcat-setenv.sh —undo
Run this commmand to recreate the symlinks, this time using the
DB2 libraries in ~bglsysdb/sqllib/java, which will work with any
level of DB2.
./tomcat-setenv.sh
Similarly update a link used by job schedulers, then update the ld
cache.
ln -sf ~bglsysdb/sqllib/lib64/libdb2.so.1 /usr/lib64/
/sbin/ldconfig
10. Update ppcfloor symlink
rm /bgl/BlueLight/ppcfloor
ln -sf /bgl/BlueLight/V1R3M4_300_2008-080728/ppc /bgl/BlueLight/ppcfloor
11. Update schema
There are no database schema updates for this release, so updating the
schema is not required.
12. The Blue Gene/L upgrade is now complete. When ready, start
bglmaster and tomcat. (You may want to specify some non-default
parameters for bglmaster. See "bglmaster —help" for more info.)
/bgl/BlueLight/ppcfloor/bglsys/bin/bglmaster start
/etc/init.d/tomcat start
BACKING OFF THE NEW RELEASE
If you want to back off the new release after it's been installed,
stop the control system and tomcat per step 9 of UPGRADE INSTRUCTIONS
and then update the symlink to point back to your previous release.
rm /bgl/BlueLight/ppcfloor
ln -s /bgl/BlueLight/<old release>/ppc /bgl/BlueLight/ppcfloor
Then start the control system and tomcat again, per step 12 of UPGRADE
INSTRUCTIONS.
DB2 VERSION 9.1 SUPPORTED
With Blue Gene/L V1R3M4, both DB2 Version 8.1 and DB2 Version 9.1
are supported on a Blue Gene/L Service Node. We are assuming that
existing customers will not choose to move to DB2 Version 9.1, since
DB2 Version 9.1 must be purchased. It is NOT a free upgrade.
If you install DB2 Version 9.1, we recommend that you also install the
most recent DB2 fixpak, which can be found at
http://www-1.ibm.com/support/docview.wss?rs=71&uid=swg21255393
DB2 Version 9 documentation is available at
http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp
Note that when migrating to a new version of DB2, you need to migrate
the instances and the databases to use the new version.
If you upgrade to DB2 Version 9.1, make certain you do step 10 of
INSTALL INSTRUCTIONS (Update symlinks to DB2 files) when you
install BG/L V1R3M4.
SPECIAL INSTRUCTIONS FOR GPFS
If GPFS is currently installed, after upgrading your Blue Gene/L
driver you must re-install GPFS by following the steps outlined in the
"Installing GPFS" section of the "GPFS for Blue Gene HOWTO". This
reinstall is required each time a new version of MCP is included with
the Blue Gene/L rpms. The "GPFS for Blue Gene HOWTO" is available in
with each version of "GPFS for Blue Gene I/O Nodes" at the Blue Gene
download website.
COMPATIBILITY OF OTHER IBM PRODUCTS WITH BLUE GENE/L V1R3M3
This chart shows the minimum level of each product that has been
tested with Blue Gene/L V1R3M4.
BGL V1R3M4
SLES 9
DB2 8.2 or 9.1
GPFS 3.1.0.22
LoadLeveler 3.4.3.1
ESSL 4.2.5.1
SAT 1.3
XLC/C++ 9.0 PTF1 (June 2008)
XLF 11.1 PTF1 (June 2008)
The base level of these products for new Blue Gene installations are
available at the Blue Gene download website:
https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=BGL-BLUEGENE
The latest versions of GPFS and LoadLeveler are available at the IBM
"Cluster Support" website:
http://www14.software.ibm.com/webapp/set2/sas/f/cluster/home.html
The latest versions of the XL Compilers are available at:
http://www-306.ibm.com/software/awdtools/xlcpp/support/
and
http://www-306.ibm.com/software/awdtools/fortran/xlfortran/support/
"ESSL for Linux on POWER V4.2.5.1" is available at the Blue Gene
download website:
https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=BGL-BLUEGENE
V1R3M4 MEMO TO USERS
The V1R3M4 "Memo to Users" is available in the Blue Gene/L KnowledgeBase at
http://www.ibm.com/eserver/iseries/support/i_dir/IBMBlueGeneLKB.nsf/wall
BUILDING THE BLRTS TOOLCHAIN
You only need to follow these instructions if you received error message
"RPM has installed but automatic building of the blrts toolchain not successful"
during the RPM install (step 2 of INSTALL INSTRUCTIONS).
From your service node, run the following commands (without the quote marks).
Step 1. Verify that the correct tar files exist in /bgl/downloads.
(if the directory does not exist you will have to create it)
Step 2. "cd /bgl/downloads"
Note: Important, the commands below need to be entered from this
directory and they need to be entered as is. The scripts use the
dirname command which require the path to be entered exactly as
specified below.
Step 3. Run the following from /bgl/downloads
"/bgl/BlueLight/V1R3M4_300_2008-080728/ppc/toolchain/retrieveToolChains.sh"
NOTE 1: You can copy this script to some other system that has
internet access as it uses curl to get the exact versions of the GNU
source necessary for BlueGene. This should pull down 4 tarball files.
NOTE 2: Leaving the 4 GNU source tarballs (binutils-2.16.1.tar.gz,
gcc-3.4.3.tar.bz2, gdb-6.4.tar.gz, glibc-2.3.6.tar.gz) in
/bgl/downloads will allow the toolchain to build automatically on your
next driver upgrade.
Step 4. Run the following from /bgl/downloads. This may take up to
40 minutes.
"/bgl/BlueLight/V1R3M4_300_2008-080728/ppc/toolchain/buildBlrtsToolChain.sh"
NOTE: This script does a checksum of the tarballs, untars, applies
patches and then builds the blrts-gnu directory in the install
directory from the RPM install.
ADDITIONAL ENHANCEMENTS AND FIXES
- All V1R3M3 eFixes have been included in V1R3M4.
- New in this release is an upgrade to the BG/L compiler.
fbaf0820ff4c5026aeb8f98c2037a640 bglbaremetal-2008.1.3-4.ppc64.rpm
0e481d125afbd54b110957baca46fb85 bglblrtstool-2008.1.3-4.ppc64.rpm
cf0d221175e761e6a6fbb09f6be3a731 bglcmcs-2008.1.3-4.ppc64.rpm
1468090c7c2185c1802873655aaa26d6 bglcnk-2008.1.3-4.ppc64.rpm
4926558b3599938ec04c30e04942f971 bgldiag-2008.1.3-4.ppc64.rpm
a0b0ab88cc6a3e37d175a3cbb5463523 bglmcp-2008.1.3-4.ppc64.rpm
8749322b64f15a5cc4b4e5820d7c4131 bglmpi-2008.1.3-4.ppc64.rpm
8a705b563768341b96c2b389fea1e0fc bglnavigator-2008.1.3-4.ppc64.rpm
2e7357706d601a420d3b34e4ccf54680 bgl_os-1.3.20080611-0.ppc64.rpm
XL C/C++ V9.0 and XL Fortran V11.1 versions are now available.
- GDB was upgraded to 6.4 level
MISCELLANEOUS ACKNOWLEDGEMENTS
fbaf0820ff4c5026aeb8f98c2037a640 bglbaremetal-2008.1.3-4.ppc64.rpm
0e481d125afbd54b110957baca46fb85 bglblrtstool-2008.1.3-4.ppc64.rpm
cf0d221175e761e6a6fbb09f6be3a731 bglcmcs-2008.1.3-4.ppc64.rpm
1468090c7c2185c1802873655aaa26d6 bglcnk-2008.1.3-4.ppc64.rpm
4926558b3599938ec04c30e04942f971 bgldiag-2008.1.3-4.ppc64.rpm
a0b0ab88cc6a3e37d175a3cbb5463523 bglmcp-2008.1.3-4.ppc64.rpm
8749322b64f15a5cc4b4e5820d7c4131 bglmpi-2008.1.3-4.ppc64.rpm
8a705b563768341b96c2b389fea1e0fc bglnavigator-2008.1.3-4.ppc64.rpm
2e7357706d601a420d3b34e4ccf54680 bgl_os-1.3.20080611-0.ppc64.rpm
Acknowledgement is given to the Association for Computing Machinery
(ACM) for permission to use the following random number generators in
Blue Gene/L diagnositics:
- #tt800.c random number generator algorithm first published in
ACM Transactions on Modelling and Computer Simulation, Vol. 4,
No. 3, 1994, pages 254-266
- Random Number Generators originally published in the article
"Efficient and Portable Combined Random Number Generators",
by Pierre L'Ecuyer, at the time Departement d'informatique,
Universite Laval, Ste Foy, Quebec, Canada; presently at Universite
de Montreal. Communications of the ACM, June 1988, Volume 31 no 6,
pp 742 ff.
This product includes software developed by the Apache Software
Foundation (http://www.apache.org/).
CHECKSUMS
If you use Download Director from the BGL download website, the
download will not complete successfully unless the checksums match,
so you don't need to check these unless you move the RPMs around
after you download them.
md5sum *.rpm
fbaf0820ff4c5026aeb8f98c2037a640 bglbaremetal-2008.1.3-4.ppc64.rpm
0e481d125afbd54b110957baca46fb85 bglblrtstool-2008.1.3-4.ppc64.rpm
cf0d221175e761e6a6fbb09f6be3a731 bglcmcs-2008.1.3-4.ppc64.rpm
1468090c7c2185c1802873655aaa26d6 bglcnk-2008.1.3-4.ppc64.rpm
4926558b3599938ec04c30e04942f971 bgldiag-2008.1.3-4.ppc64.rpm
a0b0ab88cc6a3e37d175a3cbb5463523 bglmcp-2008.1.3-4.ppc64.rpm
8749322b64f15a5cc4b4e5820d7c4131 bglmpi-2008.1.3-4.ppc64.rpm
8a705b563768341b96c2b389fea1e0fc bglnavigator-2008.1.3-4.ppc64.rpm
2e7357706d601a420d3b34e4ccf54680 bgl_os-1.3.20080611-0.ppc64.rpm