SGI Techpubs Library

Linux  »  Books  »  Administrative  »  
SGI ProPack 3 to SGI ProPack 4 for Linux Migration Guide
(document number: 007-4719-003 / published: 2006-01-24)    table of contents  |  additional info  |  download
find in page | jump to first hit | clear highlight

Chapter 1. Migrating from SGI ProPack 3 to SGI ProPack 4 for Linux

This guide contains information about migrating from SGI ProPack 3 for Linux operating system to SGI ProPack 4 for Linux Service Pack 3 operating system.


Note: The SGI ProPack 4 for Linux main download page, caveats page, and information about which configuration files you should save off before migrating to the SGI ProPack 4 release are available at the following location on Supportfolio: http://support.sgi.com/content_request/329516/index.html


SGI ProPack contains numerous libraries, utilities, and other extensions designed to supplement a base Linux distribution with additional capabilities specifically for HPC, storage, and visualization environments.

Starting with SGI ProPack 4, SGI ProPack is available to run on top of SUSE Linux Enterprise Server 9 (SLES9) Service Pack x releases (a standard, off-the-shelf Linux distribution) rather than SGI Advanced Linux Environment (an SGI built Linux distribution).

The SGI ProPack 4 for Linux SP3 release is certified and available with the 2.6 based Linux distribution of SLES 9 SP3 (or later) only. SGI ProPack 4 (and later) is not supported with other Linux distributions or the SGI Advanced Linux Environment.

About This Guide

This guide provides information about specific differences between SGI ProPack 3 and SGI ProPack 4. For general information about migrating to SUSE Linux Enterprise Server 9 (SLES9), see “SUSE Linux Migration Documentation”.

This guide contains the following sections:

Audience

This guide is written for system administrators who are responsible for installing, configuring, and administering the SGI ProPack for Linux operating system running on the SGI Altix family of servers and superclusters.

SUSE Linux Migration Documentation

The SGI ProPack 4 for Linux product includes capabilities and performance improvements ideal for enabling technical and creative users to solve their big compute and data problems by using the Linux operating system and Itanium processors. This product adds to or enhances features in SUSE Linux Enterprise Server 9 (SLES9) Service Pack 3 (SP3), the base Linux distribution based on SLES 9 SP3. SGI ProPack 4 for Linux Service Pack 3 (SP3) is designed to run on any SGI Altix system or Silicon Graphics Prism visualization system.

SUSE provides information about migrating to SLES9 at the following location: http://www.novell.com/resourcecenter/ext_item.jsp?itemId=13485.

SUSE also provides information about migrating applications to SUSE Linux at the following location: http://www.novell.com/coolsolutions/cooldev/features/a_red_hat_migration_cdev.html .

Migration Overview

This section describes general differences between SGI ProPack 3 for Linux and SGI ProPack 4 for Linux.

SGI ProPack 3 was tied to the SGI Advanced Linux Environment 3, an SGI operating system over which SGI had control. SGI ProPack 4 SP3 runs layered over SLES 9 SP3, with some additions by SGI. SGI does not provide its own version of the kernel software (for an exception to this, see “Special SGI rtgfx kernel”).

Kernel fixes and updates are provided by SUSE.

Disk devices are quite different in SGI ProPack 4 versus SGI ProPack 3. The SUSE YaST installation tool and other SUSE tools do not recognize XSCSI device names. Device names are of the /dev/sdX variety. For most SCSI and Fibre channel disks, persistent naming can be achieved by looking through /dev/disk/. Both /dev/disk and /dev/xscsi are set up using udev.

Although an XSCSI compatibility mechanism is set using udev, XSCSI paths should not be used in general except for working with Fibre channel device names.

The SLES9 kernel is far more modular than the SGI ProPack 3 kernel. In SGI ProPack 4 SP3 with SLES9 SP3 as the base system, a typical system can have upwards of 50 kernel modules loaded.

Migrating from SGI ProPack 3 for Linux

To install SGI ProPack 4 Service Pack 3 on a system running SGI ProPack 3 for Linux Service Pack 6 (or earlier), you must do a complete re-installation on a clean disk. Before attempting the SGI ProPack 4 SP3 installation, perform the following:

  1. Verify PROM firmware is 4.43 or higher. To see the PROM version currently on the system, use the following command:

    cat /proc/sgi_prominfo/*/version

    If you find that you do need to update your PROM, you can download PROM 4.43 or the latest PROM directly from Supportfolio Online: https://support.sgi.com/browse_request/suse_patch_browse_results

    Once you have the PROM RPM, follow the steps outlined in the section entitled "Obtaining the Latest SGI Altix System Firmware" in the SGI ProPack 4 for Linux Start Here to update the PROM on your system. If a printed version of this manual is not readily available, a PDF version is available in the /docs directory on the SGI ProPack 4 SP3 CD1 and online at http://docs.sgi.com.

  2. Verify that all bricks on the system are running the same revision of L1 firmware. You can always download the latest L1/L2 system controller software from Supportfolio Online: http://support.sgi.com/linux/

  3. Save any configuration files you may have modified. To locate modified configuration files, enter the following command:

    rpm -V -a | grep "\b c"

    Save these files on a disk other than the one where SGI ProPack will be installed. These files can serve as a valuable reference after installation if you plan to make similar changes.

  4. Save any FLEXlm license files that might be necessary to support licensed applications after the SGI ProPack 4 SP3 installation.

  5. Save off any other files or data that may exist on the local disk where the SGI ProPack 4 SP3 release is to be installed.

  6. Disconnect any non-system drives from your system. This will help speed up the installation since the SLES install will not scan those drives. This also helps avoid accidental data deletion.

  7. Proceed to the “How To Install SGI ProPack 4 for Linux SP3 For the First Time” instructions in the SGI ProPack 4 for Linux Start Here.

Differences between SGI ProPack 3 and SGI ProPack 4

This section describes some important differences between SGI ProPack 3 and SGI ProPack 4 and covers these topics:

Latest PROM Firmware

Unless you have the most current PROM, the kernel may fail to boot during the installation procedure. The prom RPM is on ProPack 4 Proprietary CD in SGI/RPMS directory.


Note: Systems running SLES9 SP3 and SGI ProPack 4 SP3 should use PROM 4.43 which ships with SGI ProPack 4. The updated PROM contains key bug fixes, along with support for new kernel features, such as memory error recovery. Note that PROM 4.43 and subsequent versions are not compatible with SLES9 SP1. SGI Altix or Silicon Graphics Prism software support contract customers can download PROM version 4.43 or later from the following area on Supportfolio: https://support.sgi.com/browse_request/suse_patch_browse_results


For information on obtaining the latest PROM, see the “Obtaining the Latest SGI Altix System Firmware” section in Chapter 2, “Software Planning and Installation”, in the SGI ProPack 4 for Linux Start Here.

Topology

Topology information is available in /proc/sgi_sn/sn_topology . /dev/hw is not available on ProPack 4 systems. You may access this information in several different formats using /usr/bin/topology or /usr/bin/gtopology .

Booting a SLES9 SP3 System

Use efi/boot/bootia64 instead of elilo to boot from the SLES9 or SLES9 SP3 CDs (no boot option should be applied and any other boot option like console=ttyS0 should be placed in the input field of the boot menu that appears after bootia64 has been started). Once the system is running, SLES9 SP3 (and/or SGI ProPack 4 SP3), use elilo to boot from efi.

If you have installed multiple kernel images and you want to boot with one that is not currently the system default (vmlinuz in /boot/efi/efi/SuSE), simply copy the vmlinuz and initrd files from the /boot directory to /boot/efi/efi/SuSE directory.

You should never change the /boot/efi/efi/SuSE/elilo.conf file directly. Instead, you can change the /etc/elilo.conf file and run the elilo command. This helps prevent the installer software from clobbering your changes.

On SGI systems running SGI ProPack 4 for Linux, use bootia64 rescue console=ttyS0 command when booting in rescue mode. For more information, see the “Recovering a Damaged Root Filesystem” section in the Linux Configuration and Operations manual.


Note: On SGI Prism visualizaton systems, if you have a bootable CD in the DVD drive when the system is reset and EFI Shell menu choice is selected, the CD is assigned the file system number fs0, at the top of the list. On an SGI Altix system, it is assigned fs1 at the bottom of the list.


Linux Threads Behavior

With SGI ProPack 3, to use the old Linux threads pthreads model, you needed to set the LD_ASSUME_KERNEL variable, as follows

export LD_ASSUME_KERNEL=2.4.19

With SGI ProPack 4 SP3 on SLES9 SP3, to get the same thread behavior, set the LD_ASSUME_KERNEL variable, as follows:

export LD_ASSUME_KERNEL=2.4.21

This is documented in the SLES9 release notes. Applications compiled on SGI ProPack 3 (and some compiled on SGI ProPack v2.4) will run with this environment variable set. For more information about applications built on SGI ProPack for Linux using the Native Posix Thread Library, see the “Profiling with profile.pl”in the Linux Application Tuning Guide.

Kernel Placement

In SGI ProPack 3, when you installed a new kernel with the rpm -ivh command, the new kernel would appear in the /boot/efi/efi/sgi directory but would not overwrite old kernels already there. With SLES9, if you install a kernel with rpm -ivh, the old kernel is wiped out in /boot/efi/efi/SuSE directory. There is only one kernel and only one initrd(4) boot loader block device stored in the /boot/efi/efi/SuSE directory. The rpm listing will still show multiple kernels and they will still be available in the /boot directory. However, elilo will only be able to find the "latest kernel you installed" at this time. This is also how YaST/YaST2 installer handles kernel installation. For more information about booting different installed kernels, see “Booting a SLES9 SP3 System”.

User IDs/Group IDs

SGI ProPack 3 user/group IDs start at 500 in the /etc/login.defs file. SUSE user/group IDs start at 1000 in the /etc/login.defs file. If you are running both SGI ProPack 3 and SLES9 SP3 and SGI ProPack 4 on the same system, set the UIDs to 1000 in SGI ProPack 3 to avoid problems when switching between SGI ProPack 3 and SGI ProPack 4.

CpuMemSets

CpuMemSets functionality in SGI ProPack 3 for Linux is contained within the new cpuset implementation on SGI ProPack 4. For more information, see “Cpusets”.

Cpusets

The cpuset implementation is new on SGI ProPack 4. For detailed information, see Chapter 6, “Cpusets on SGI ProPack 4 for Linux” in the Linux Resource Administration Guide. You can also find information at the following location:

/usr/share/doc/packages/cpuset/cpuset.html

Installing and Configuring Array Services

An account must exist on all hosts in the array for the purposes of running certain Array Services commands. This is controlled by the /etc/sysconfig/array configuration file. Note that the configuration file is /etc/config/arrayd.options in SGI ProPack 3 releases. The default is to use the user account arraysvcs on SGI ProPack 4 systems. The arraysvcs user account should be created by default during installation. The user account guest is used on SGI ProPack 3 systems. The account name can be changed in arrayd.conf. For more information, see the arrayd.conf(5) man page and Chapter 3, “Array Services in the Linux Resource Administration Guide.

Comprehensive System Accounting

In Comprehensive System Accounting (CSA), user exits allow you to tailor the csarun or csaperiod procedures to the specific needs of your site by creating scripts to perform additional site-specific processing during daily accounting. You need to create user exit files owned by user csaacct (this was user adm for SGI ProPack 3 ) with execute permission if your site uses the accounting user exits. The csaacct user account is created by default during installation. For more information, see Chapter 2, “Comprehensive System Accounting” in the Linux Resource Administration Guide.

Partitioning Software

On SGI ProPack 4 systems, if you intend to use the cross-partition functionality of XPMEM, you will need to add xpc to the line in the /etc/sysconfig/kernel file that begins with MODULES_LOADED_ON_BOOT. Once that is added, you may either reboot the system or issue a modprobe xpc command to get the cross-partition functionality to start working. For more information on using modprobe, see the modprobe(8) man page. For more information on partitioning software, see the “System Partitioning” section in the Linux Configuration and Operations Guide.

For the SGI ProPack 4 for Linux SP3 release, the following system configuration guidelines are valid:

  • Maximum number of partitions supported is 48

  • Maximum partition size is 512 processors

  • Maximum system size is 2048 processors

For additional information about configurations that are supported for system partitioning, see your sales representative. SGI field support personnel may reference the SGI Altix 3000 Internal Technical Configuration Manual.

Persistent IP Addressing of Ethernet Interfaces

The mechanism for associating persistent IP addresses with Ethernet interfaces is different in SLES9 SP3 than it was in SGI ProPack 3. In particular, the /etc/sysconfig/networking/eth0_persist file is no longer used. For information on Persistent IP Addressing of Ethernet Interfaces in SGI ProPack 4, see Chapter 3 of the SGI ProPack 4 for Linux Start Here.

runon Command

The runon(1) command is a deprecated package on SGI ProPack 4. You can use the taskset (1) command to retrieve or set the CPU affinity of a process. For more information, see the taskset(1) man page.

KDB Kernel Debugger sn2kdb command

The sn2kdb command has been replaced with the archkdb command.

Hardware Information

The Altix hardware inventory command, hinv(1) is not available on SLES9 SP3 with SGI ProPack 4 SP3. Instead, please use the SUSE hwinfo command ( hwinfo --short, hwinfo --disk, and sitar --format=html).


Note: The hwinfo command can take a very long time to complete when run as root. Run as a non-root user for quicker results.


For more information, see the hwinfo(8) man page and information in /usr/share/doc/packages/hwinfo.

xfsdump/xfsrestore Commands

xfsdump(8) and xfsrestore(8) are not loaded by default. Install the xfsdump and xfsprogs-devel rpms if you need these commands.

YaST Installer

The YaST installer does not allow you to remove older partitions and create a clean root by default. To do so, use Expert mode. The default filesystem type is reiserfs. To select XFS, you will need to select Edit to edit the partition. Boot partitions should use the FAT filesystem.

With the exception of fdisk, xvm, and some sysctl options, it is best to let YAST handle options instead of trying to modify files by hand.

Per-Node Memory Available Statistics

The per-node memory available statistics that were available in /proc/discontig are now part of sysfs. For example:

% cat /sys/devices/system/node/node0/{meminfo,numastat}

Node 0 MemTotal:      1972384 kB
Node 0 MemFree:       1294912 kB
Node 0 MemUsed:        677472 kB
Node 0 HighTotal:           0 kB
Node 0 HighFree:            0 kB
Node 0 LowTotal:      1972384 kB
Node 0 LowFree:       1294912 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:    262144 kB
numa_hit 3070703
numa_miss 0
numa_foreign 0
interleave_hit 1
local_node 3069479
other_node 1224

The following is text for the new numa.mem metrics:

numa.mem.util.total [per-node total memory]
numa.mem.util.free [per-node free memory]
numa.mem.util.used [per-node used memory]
numa.mem.util.hightotal [per-node highmem total]
numa.mem.util.highfree [per-node highmem free]
numa.mem.util.lowtotal [per-node lowmem total]
numa.mem.util.lowfree [per-node lowmem free]
numa.mem.util.hugepages_total [per-node hugepages total]
numa.mem.util.hugepages_free [per-node hugepages free]
numa.mem.util.hugepagessize [per-node hugepage size]
numa.mem.alloc.numa_hit [per-node count of number of times a process 
  wanted to alloc memory on the local node and succeeded]
numa.mem.alloc.numa_miss [per-node count of times a process wanted to 
  alloc on the local node but got memory on another node]
numa.mem.alloc.numa_foreign [count of times a process on another node 
  wanted to alloc on that node, but got this node]
numa.mem.alloc.interleave_hit [count of the number of time interleaving 
  wannted to allocate on this node and succeeded]
numa.mem.alloc.local_node [count of times a process ran on this node 
  and got memory on this node]
numa.mem.alloc.other_node [count of times a process ran on this node 
  and got memory from another node]

Kernels

The SGI ProPack CDs no longer contain any kernel packages. SLES9 SP3 contains several different kernels and they all ship as part of the SLES9 SP3 set of ISO images. This section gives a brief explanation of the differences between each kernel. It covers these kernels:

kernel-sn2-2.6.5-xxx

You must explicitly select the sn2 kernel for installation during the install of SLES9/SLES9 SP3.


Note: This manual uses generic names for the different kernels shipped with SGI ProPack 4. The specific version of the kernel shipped on your system is similar to kernel-sn2-2.6.5-7.201



Note: Both the kernel-default ("generic") and kernel-sn2 ("sn2") kernels are certified and supported on SGI Altix systems; however, SGI recommends using the sn2 kernel in most cases since this kernel is configured to support larger single-system images (maximum 512 processors) . Both the kernel-default ("generic") and kernel-sn2 ("sn2") kernels include the kernel debugger (that is, KDB) to aid customer support should that become necessary.


The kernel-sn2-2.6.5-xxx (or later) kernel has these characteristics:

  • sn2 specific kernel

  • Maximum 512 CPU support

  • 16KB page size

  • KDB support (must be configured on)

kernel-64k-pagesize-2.6.5-xxx

The kernel-64k-pagesize-2.6.5-xxx (or later) kernel has these characteristics:

  • ia64 generic kernel

  • Maximum 128 CPU support

  • 64KB page size

  • No KDB support

  • Does not work with XFS filesystems of 512 byte blocks, but does work if XFS filesystem is created with larger blocks

kernel-default-2.6.5-xxx

The kernel-default-2.6.5-xxx (or later) kernel has these characteristics:

  • ia64 generic kernel

  • Maximum 128 CPU support

  • 16KB page size

  • KDB support (must be configured on)

kernel-debug-2.6.5-xxx

The kernel-debug-2.6.5-xxx (or later) kernel has these characteristics:

  • ia64 generic kernel

  • Maximum 512 CPU support

  • 16KB page size

  • KDB support (enabled by default)

  • kprobes support

  • dprobes support

  • slab/spinlock debugging enabled

Special SGI rtgfx kernel

If you are installing the graphics, dmedia, or REACT groups from the YaST install program, the kernel you will be using is the SGI built rtgfx kernel. Also, the associated modules will be auto-selected if you choose SGI ProPack Graphics, SGI ProPack Digital Media, and/or SGI REACT.


Caution: Because the rtgfx kernel is not a standard SUSE kernel variant, it is not supported in any way by SUSE or Novell. Installation of the rtgfx kernel will also invalidate any existing system certifications with SLES9 SP3. Only users who require support for digital media, graphics, or real-time should install any of the software from the System Extensions CD.

For more information, see Chapter 2, “Software Planning and Installation”in the SGI ProPack 4 for Linux Start Here .


Workarounds

This section describes important workarounds when migrating to SGI ProPack 4 for Linux.

XVM Disks with GPT labels

Disks that are shipped with SGI Altix systems may be pre-partitioned with GPT labels. GPT partition tables, often created by operating system installers or the parted partitioning tool, store labels in two locations. A GPT labeled disk contains primary partition information at the beginning of the disk and backup information at the end of the disk. Old PROM versions can unlabel disks and also some operating system install/update utilities can unlabel disks.

When using XVM, you must be sure a backup GPT label does not exist. A script (xvmfixlabel) is available on Supportfolio that scans every disk in the system and removes GPT label information from those disks with SGI labels. To access the script perform the following steps:

To access the script perform the following steps:

  1. Point your browser at http://support.sgi.com and login to Supportfolio.

  2. Select Downloads

  3. Enter the keyword xvmfixlabel

Dedicated time is not required to run the script. You must be logged in as root when executing the script. The script requires that the C compiler is installed on your system. Execute the script, as follows:

csh ./xvmfixlabel

Tools and Library Changes

This section describes changes to tools and libraries between the SGI ProPack 3 and SGI ProPack 4 releases that may impact application develorpers, as follows:

  • glut3.7 libraries have changed to the freeglut2.2 libraries.

    The freeglut libraries do not contain Fortran bindings for glut codes.

  • SGI ProPack 4 does not support Fortran bindings for OpenGL.

Major Documentation Changes for SGI ProPack 4 Service Pack 3

For information on manuals that are new or updated for the SGI ProPack 4 for Linux Service Pack 3 release, see the “Documentation Updated for This Release” section of Chapter 1, “Release Features” in the SGI ProPack 4 for Linux Start Here.

Obtaining Publications

To obtain SGI documentation, go to the SGI Technical Publications Library at http://docs.sgi.com.

Reader Comments

If you have comments about the technical accuracy, content, or organization of this document, please tell us. Be sure to include the title and document number of the manual with your comments. (Online, the document number is located in the front matter of the manual. In printed manuals, the document number can be found on the back cover.)

You can contact us in any of the following ways:

  • Send e-mail to the following address:

    techpubs@sgi.com

  • Use the Feedback option on the Technical Publications Library World Wide Web page:

    http://docs.sgi.com

  • Contact your customer service representative and ask that an incident be filed in the SGI incident tracking system.

  • Send mail to the following address:

    Technical Publications
    SGI
    1500 Crittenden Lane, M/S 535
    Mountain View, California 94043-1351

We value your comments and will respond to them promptly.

SGI ProPack 3 to SGI ProPack 4 for Linux Migration Guide
(document number: 007-4719-003 / published: 2006-01-24)    table of contents  |  additional info  |  download

    Front Matter
    Chapter 1. Migrating from SGI ProPack 3 to SGI ProPack 4 for Linux


home/search | what's new | help