SGI Techpubs Library

IRIX 6.5  »  Books  »  Administrative  »  
CXFS MultiOS Client-Only Guide for SGI InfiniteStorage
(document number: 007-4507-016 / published: 2008-04-16)    table of contents  |  additional info  |  download
find in page

Chapter 3. AIX Platform

CXFS supports a client-only node running the AIX operating system. This chapter contains the following sections:

CXFS on AIX

This section contains the following information about CXFS on AIX:

Requirements for AIX

In addition to the items listed in “Requirements” in Chapter 1, using an AIX node to support CXFS requires the following:

  • IBM AIX 5L: Version 5.3 Maintenance Level 3 (64-bit mode) APAR number IY71011 or its successor

    To verify the operating system level, use the following command:

    oslevel -r

  • IBM FC5716, FC6228, or FC6239 2-Gbit Fibre Channel host bus adapters (HBAs)

  • One or more of the following IBM hardware platforms:

    pSeries 570
    pSeries 575
    pSeries 595
    pSeries 610
    pSeries 620
    pSeries 630
    pSeries 640
    pSeries 650
    pSeries 660
    pSeries 670
    pSeries 680
    pSeries 690

For the latest information, see the CXFS AIX release notes.

CXFS Commands on AIX

The following commands are shipped as part of the CXFS for AIX package:

/usr/cxfs_cluster/bin/cxfs_client
/usr/cxfs_cluster/bin/cxfs_info
/usr/cxfs_cluster/bin/grioadmin
/usr/cxfs_cluster/bin/grioqos
/usr/cxfs_cluster/bin/xvm

The cxfs_client and xvm commands are needed to include a client-only node in a CXFS cluster. The cxfs_info command reports the current status of this node in the CXFS cluster.

The lslpp output lists all of the software added; see “Installing CXFS Software on AIX”.

For more information on these commands, see the man pages. For information about the GRIO commands, also “Guaranteed-Rate I/O (GRIO) and CXFS” in Chapter 1 and “GRIO on AIX”.

Log Files on AIX

The cxfs_client command creates a /var/tmp/cxfs_client log file. To rotate this log file, use the -z option in the following file:

/usr/cxfs_cluster/bin/cxfs_client.options

See the cxfs_client man page for details.

Some daemons related to CXFS output a message in the console log. To see the contents of this log file, use the following command:

alog -o -t console

The console log is rotated.

For information about the log files created on server-capable administration nodes, see the CXFS Administration Guide for SGI InfiniteStorage.

Also see the AIX /etc/syslog.conf file.

CXFS Mount Scripts on AIX

AIX supports the CXFS mount scripts. See “CXFS Mount Scripts” in Chapter 1 and the CXFS Administration Guide for SGI InfiniteStorage.

Limitations and Considerations for AIX

Note the following:

  • Although it is possible to mount a JFS or NFS filesystem on top of an AIX CXFS filesystem, this is not recommended.

  • There is no default access control list (ACL) in AIX. Therefore, the setup and display of the default ACL cannot be completed using the following commands:

    aclget
    aclput
    acledit

    If an IRIX ACL exists, the ACL becomes effective when the default ACL is set up by IRIX and a file and a directory are made under that directory in AIX.

  • There is no MASK entry in AIX, but the access permissions in AIX follow those established when an ACL set up by Linux contains a MASK entry. If the default ACL is set up for a given directory and the MASK entry exists, then that MASK entry is used when a file or a subdirectory is made by AIX. When the MASK entry does not exist, rwx is used.

  • ACL control of the following, which the AIX JFS filesystem has, cannot be applied to CXFS:

    • The access to a certain user or the group is rejected (deny)

    • When a user belongs to the specific group, access is permitted or rejected (specify)

    If deny or specify is used, an error occurs (EINVAL) because these features are not in CXFS.

  • Socket files cannot be copied. The following error is returned:

    AIX:The socket does not allow the requested operation. 

  • You can use the fuser command to extract process information about the mounted filesystem, but you cannot extract process information about the file or the directory.

  • When a CXFS mount is performed on a mirror volume created by XVM, the AIX system goes into panic status. The mirror volume cannot be mounted on the AIX CXFS system.

  • The AIX node does not automatically detect the worldwide port number (WWPN). In order to use I/O fencing, you must list the WWPN in the /etc/fencing.conf file. See “License Keys” in Chapter 1.

  • If your users want to use a file size/offset maximum greater than 1 GB, you must change their user properties to allow files of unlimited size. To do this, use the smit command. For more information, see the smit man page.

  • By default, the maximum request size for direct I/O is 512 MB (524288 KB). A direct I/O request larger than 512 MB will revert to buffered I/O. However, you can change the maximum XVM direct memory access (DMA) size to improve direct I/O performance. To do this, use the chdev command to modify the xvm_maxdmasz attribute. The actual maximum limit will always be 4 KB less than any of the supplied or displayed values (for example, the default is actually 512 MB minus 4 KB).


    Note: The XVM module must be loaded if any attribute changes are to be noticed and applied.

    To display the current setting, use the following command:

    lsattr -E -l xvm -a xvm_maxdmasz

    To change the current setting, use the following command:

    chdev [-P|-T] -l xvm -a xvm_maxdmasz=NewValue

    Legal values for NewValue are specified in KB units in the range 256 to 2097152 (that is, 256 KB to 2 GB).

    By default, using chdev on a running system makes a permanent change for subsequently mounted filesystems. (Running filesystems will not be changed until they are remounted, either manually or after a reboot.)

    If you use -P, the change is deferred until the next boot and after that it is permanent. If you use -T (temporary), the change is immediate for subsequently mounted filesystems, but lasts only until the next boot.

    For example, to change the DMA size to 2 GB for subsequently mounted filesystems on the currently running device and in the database, enter the following:

    aix# chdev -l xvm -a xvm_maxdmasz=2097152

    For more information, see the lsattr and chdev man pages.

  • Due to FC controller limitations, large ( > 256K) direct I/O requests may be problematic.

  • If you use the frametest command on AIX, make sure that the posix_aio0 device is available. Do the following:

    1. Change the setting of the posix_aio0 device to available:

      aix# chdev -l posix_aio0 -a autoconfig=available

    2. Add the device to the system:

      aix# mkdev -l posix_aio0

    3. Verify that the device is available. For example:

      aix# lsdev|grep aio
      aio0       Defined                 Asynchronous I/O (Legacy)
      posix_aio0 Available               Posix Asynchronous I/O

    For more information about the AIX commands, see their man pages.

    For an overview of frametest, see the section about generation of streaming workload for video streams in the CXFS Administration Guide for SGI InfiniteStorage. For details about frametest and its command-line options, see the frametest (1) man page.

See also Appendix B, “Filesystem and Logical Unit Specifications”.

Maximum CXFS I/O Request Size and AIX

By default, the maximum CXFS I/O request size for normal filesystem I/O is 1 MB (1024 KB). However, depending on filesystem size and internal layout, the actual request size can be smaller or larger:

  • Requests that are smaller than 1 MB are unaffected by the limit and proceed normally

  • Requests larger than 1 MB are automatically split into multiple smaller requests in order to accommodate the limit

The cxfs_maxiosz attribute determines the CXFS maximum I/O size request. To display the current setting, use the lsattr command. For example:

aix# lsattr -E -l xvm -a cxfs_maxiosz

To change the CXFS maximum I/O request size, use the chdev command to modify the cxfs_maxiosz attribute. For example:

aix# chdev [-P|-T] -l xvm -a cxfs_maxiosz=NewValue


Note: For attribute changes to be noticed and applied, the XVM module must be loaded.

Legal values for NewValue are specified in KB units in the range 64 through 2048 (that is, 64 KB to 2 MB).

By default, using chdev on a running system makes a permanent change for subsequently mounted filesystems. (Running filesystems will not be changed until they are remounted, either manually or after a reboot.)

If you use -P, the change is deferred until the next boot and after that it is permanent. If you use -T (temporary), the change is immediate for subsequently mounted filesystems, but lasts only until the next boot.

For example, to change the CXFS maximum I/O request size to 512 KB for subsequently mounted filesystems on the currently running device xvm and in the database, enter the following:

aix# chdev -l xvm -a cxfs_maxiosz=512

For more information, see the lsattr and chdev man pages.

There is a possibility that CXFS I/O limits may conflict with AIX's internal disk driver limits. In such cases, you will see console error messages from CXFS that specify an illegal request size error. You can use one of the following ways to correct this problem:

  • You can decrease CXFS maximum I/O size to match the limit imposed by the AIX disk driver using a procedure similar to the above. This AIX limit is per physical disk drive and is described by the AIX attribute max_transfer. You can display this limit with the lsattr command if you know the name of the physical disk that corresponds to your XVM volume. For example, where hdiskXX is the subsystem name that AIX chooses for each physical disk driver it finds at boot time (the XX number will vary depending upon controller configuration and number of drives):

    aix# lsattr -E -l hdiskXX -a max_transfer
    max_transfer 0x40000 Maximum TRANSFER Size True

    The hexadecimal value 0x40000 is 256 KB. From the CXFS error messages on the console, you can find the transfer size that CXFS tried to use; it will likely be hexadecimal 0x80000 (512 KB), which is too large. You can decrease the CXFS maximum I/O size to 256 KB to match AIX's max_transfer limit. This decrease may slightly decrease overall filesystem performance.

  • You can increase AIX's per-physical-disk max_transfer attribute to 512 KB to match the CXFS maximum I/O request size. You must perform the following command for each physical disk that is part of the cluster configuration:

    aix# chdev -l hdiskXX -a max_transfer=0x80000

    You can verify the change by using lsattr command as described above.

After modifying AIX's disk driver limits, you must reboot the machine to allow the changes to take effect.

Access Control Lists and AIX

All CXFS files have UNIX mode bits (read, write, and execute) and optionally an ACL. For more information, see the AIX chmod, acledit, aclget, and aclput man pages.

If you want to use an AIX node to restore a CXFS file with an ACL, you should use the backup and restore commands. If you use the tar, cpio, or pax command, the ACL will not be used because these tools behave "intelligently" by not calling acl subroutines to set an ACL. These tools will only set the file mode.

When using the ls command to display access permissions for a file with an ACL, the mode reported for a CXFS file follows Linux semantics instead of AIX JFS semantics.

The CXFS model calls for reporting the ACL MASK for the group permission in the mode. Therefore, if the GROUP entry is r-x and the MASK entry is rw-, the group permission will be reported as rw-. Although it appears that the group has write permission, it does not and an attempt to write to the file will be rejected. You can obtain the real (that is, effective) group permission by using the AIX aclget command.


Note: Normally, AIX filesystem ACLs can have up to one memory page (4096 bytes) for a file and a directory. However, CXFS filesystems on AIX nodes in a multiOS cluster must maintain compatibility with the metadata server. The CXFS filesystems on an AIX node are limited to a maximum of 25 ACL entries converted to Linux ACL type for a file and a directory.


Storage Partitioning and XVM Failover V2 for AIX

IBM hosts running the AIX 5L operating system set the QERR mode page bit to 1 for support storage (other than IBM storage), which does not work well with SGI ProPack for Linux server-capable administration nodes: Linux will leave CTQ enabled but suffer from timeouts.

There is an administrative work-around for this problem. Engenio offers an enhanced feature called SANshare for storage partioning. There is an additional licensing cost required to obtain a SANshare license. SANshare allows hosts to be grouped separately and still access the same LUNs, thus allowing the IBM AIX 5L hosts to set the QERR mode page bit to 1 and not affect the other hosts accessing the LUN.

For each RAID unit, create one Host Group for all of the AIX 5L systems and the other hosts in the CXFS cluster. Set the Host Type to LINUX for the AIX nodes and to SGIAVT for the other nodes. (The Host Type of AIX has AVT status disabled.)

HBA Installation for AIX

For information about installing and configuring the host bus adapter (HBA), see the IBM HBA documentation.

Preinstallation Steps for AIX

This section provides an overview of the steps that you or a qualified IBM service representative will perform on your AIX nodes prior to installing the CXFS software. It contains the following sections:

Adding a Private Network for AIX

The following procedure provides an overview of the steps required to add a private network to the AIX system. A private network is required for use with CXFS. See “Use a Private Network” in Chapter 2.

You may skip some steps, depending upon the starting conditions at your site. For details about any of these steps, see the AIX documentation.

  1. If your system is already operational and on the network, skip to step 2. If the AIX operating system has not been installed, install it in accordance with the AIX documentation.

  2. Edit the /etc/hosts file so that it contains entries for every node in the cluster and their private interfaces.

    The /etc/hosts file has the following format, where primary_hostname can be the simple hostname or the fully qualified domain name:

    IP_address      primary_hostname      aliases

    You should be consistent when using fully qualified domain names in the /etc/hosts file. If you use fully qualified domain names on a particular node, then all of the nodes in the cluster should use the fully qualified name of that node when defining the IP/hostname information for that node in the /etc/hosts file.

    The decision to use fully qualified domain names is usually a matter of how the clients (such as NFS) are going to resolve names for their client server programs, how their default resolution is done, and so on.

    Even if you are using the domain name service (DNS) or the network information service (NIS), you must add every IP address and hostname for the nodes to /etc/hosts on all nodes.

    For example:

    190.0.2.1 server1.company.com server1
    190.0.2.3 stocks 
    190.0.3.1 priv-server1 
    190.0.2.2 server2-.company.com server2
    190.0.2.4 bonds 
    190.0.3.2 priv-server2

    You should then add all of these IP addresses to /etc/hosts on the other nodes in the cluster.


    Note: Exclusive use of NIS or DNS for IP address lookup for the nodes will reduce availability in situations where the NIS or DNS service becomes unreliable.

    For more information, see “Understand Hostname Resolution and Network Configuration Rules” in Chapter 2 and the hosts, named, and nis man pages.

  3. (Optional) Edit the /etc/netsvc.conf file so that local files are accessed before either NIS or DNS. That is, the hosts line in /etc/netsvc.conf must list local first. For example:

    hosts = local,nis,bind 

    (The order of nis and bind is not significant to CXFS, but local must be first.)

  4. Determine the name of the private interface by using the ifconfig command as follows, to list the available networks. For example:

    # ifconfig  -l
    en0 en1 lo0

    However, if the second network interface (en1) does not appear, then the network interface must be set up in accordance with the AIX documentation.

    You can set up an IP address by using ifconfig after restarting the system. If it is set up properly, the following information is output (line breaks added here for readability):

    #  ifconfig  -a
    en0:flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG>
       inet 10.208.148.61 netmask 0xffffff00 broadcast 10.208.148.255
    en1:flags=7e080863,10<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRPT,64BIT,
       CHECKSUM_OFFLOAD,CHECKSUM_SUPPORT,RSEG>
       inet 192.168.10.61 netmask 0xffffff00 broadcast 192.168.10.255
    lo0:flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT>
       inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 

  5. (Optional) Edit the /.rhosts file if you want to use remote access or if you want to use the connectivity diagnostics with CXFS. Make sure that the mode of the .rhosts file is set to 600 (read and write access for the owner only).

    Make sure that the /.rhosts file on each AIX node allows all of the nodes in the cluster to have access to each other. The connectivity tests execute a ping command from the local node to all nodes and from all nodes to the local node. To execute ping on a remote node, CXFS uses rsh as user root.

    For example, suppose you have a cluster with three nodes: propack0, aix1, and aix2. The /.rhosts files could be as follows (where the prompt denotes the node name):

    propack0# cat /.rhosts 
    aix1 root
    aix2 root
          
    aix1# cat /.rhosts
    propack0 root
    aix2 root
          
    aix2# cat /.rhosts
    propack0 root
    aix1 root

Verifying the Private and Public Network for AIX

For each private network on each AIX node in the pool, verify access with the AIX ping command. Enter the following, where nodeIPaddress is the IP address of the node:

/usr/sbin/ping  -c 3  nodeIPaddress

For example:

aix# /usr/sbin/ping -c 3 192.168.10.61 
PING  192.168.10.61: (192.168.10.61): 56 data data bytes
64 bytes from 192.168.10.61  icmp_seq=0 ttl=255 time=0 ms 
64 bytes from 192.168.10.61  icmp_seq=1 ttl=255 time=0 ms 
64 bytes from 192.168.10.61  icmp_seq=2 ttl=255 time=0 ms 
----192.168.10.61 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/0/00 ms

You should also execute a ping on the public networks. If that ping fails, follow these steps:

  1. Verify that the network interface was configured up. For example:

    aix# /usr/sbin/ifconfig en0 
    en0: flgs=4e08086<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG> 
      inet 10.208.148.61 netmask 0xffffff00 broadcast 10.208.148.255

    In the first output line above, UP indicates that the interface was configured up.

  2. Verify that the cables are correctly seated. Repeat this procedure on each node.

Client Software Installation for AIX

The CXFS software initially will be installed and configured by SGI personnel. This section discusses the following:

Installing CXFS Software on AIX

Installing CXFS for AIX requires approximately 20 MB of space. To install the required software on an AIX node, SGI personnel will do the following:

  1. Read the SGI InfiniteStorage Software Platform release notes, CXFS general release notes, and CXFS AIX release notes from the /docs directory on the ISSP DVD and any late-breaking caveats on Supportfolio.

  2. Verify that the node has been upgraded to the supported AIX version according to the AIX documentation. Use the following command to display the currently installed system:

    oslevel -r

    For example, the following output indicates AIX version 5, revision 3, maintenance level 03:

    aix# oslevel -r
    5300-03

  3. Transfer the client software that was downloaded onto a server-capable administration node during its installation procedure using ftp, rcp, or scp . The location of the tarball on the server will be as follows:

    /usr/cluster/client-dist/CXFS_VERSION/aix/53ml3/noarch/SGIcxfs-aix5L

  4. Install the CXFS software (the example output below is truncated):

    aix# installp -a -d SGIcxfs-aix5L all
    +-----------------------------------------------------------------------------+
                        Pre-installation Verification...
    +-----------------------------------------------------------------------------+
    Verifying selections...done
    Verifying requisites...done
    Results...
    
    SUCCESSES
    ---------
      Filesets listed in this section passed pre-installation verification
      and will be installed.
    
      Selected Filesets
      -----------------
      SGIcxfs-aix5L 5.0.0.5                       # CXFS CLIENT for AIX
    
      << End of Success Section >>
    
    FILESET STATISTICS
    ------------------
        1  Selected to be installed, of which:
            1  Passed pre-installation verification
      ----
        1  Total to be installed
    
    +-----------------------------------------------------------------------------+
                             Installing Software...
    +-----------------------------------------------------------------------------+
    
    installp:  APPLYING software for:
            SGIcxfs-aix5L 5.0.0.5
    
    
    . . . . . << Copyright notice for SGIcxfs-aix5L >> . . . . . . .
    ...
    
    Finished processing all filesets.  (Total time:  4 secs).
    
    +-----------------------------------------------------------------------------+
                                    Summaries:
    +-----------------------------------------------------------------------------+
    
    Installation Summary
    --------------------
    Name                        Level           Part        Event       Result
    -------------------------------------------------------------------------------
    SGIcxfs-aix5L               5.0.0.5         USR         APPLY       SUCCESS
    SGIcxfs-aix5L               5.0.0.5        ROOT        APPLY       SUCCESS
    

  5. Reboot to start CXFS services automatically.

Verifying the AIX Installation

To verify that the CXFS software has been installed properly, use the lslpp command as follows:

aix# lslpp -L SGIcxfs-aix5L

For example, the following output (showing a state of C, for “committed”) indicates that the CXFS package installed properly:

aix# lslpp -L SGIcxfs-aix5L
Fileset                      Level  State  Type  Description (Uninstaller)
  ----------------------------------------------------------------------------
  SGIcxfs-aix5L              5.0.0.5   C     F    CXFS CLIENT for AIX 


State codes: 
 A -- Applied. 
 B -- Broken. 
 C -- Committed. 
 E -- EFIX Locked.
 O -- Obsolete.  (partially migrated to newer version) 
 ? -- Inconsistent State...Run lppchk -v.

Type codes: 
 F -- Installp Fileset 
 P -- Product 
 C -- Component 
 T -- Feature 
 R -- RPM Package

I/O Fencing for AIX

I/O fencing is required on AIX nodes in order to protect data integrity of the filesystems in the cluster. The /etc/fencing.conf file enumerates the worldwide port name (WWPN) for all of the host bus adapters (HBAs) that will be used to mount a CXFS filesystem. The /etc/fencing.conf file must contain a simple list of WWPNs as 64-bit hexadecimal numbers, one per line. These HBAs will then be available for fencing.

If you want to use the /etc/fencing.conf file, you must update it whenever the HBA configuration changes, including the replacement of an HBA.

Do the following:

  1. Follow the Fibre Channel cable on the back of the AIX host to determine the port to which it is connected in the switch. Ports are numbered beginning with 0. (For example, if there are 8 ports, they will be numbered 0 through 7.)

  2. Use the telnet command to connect to the switch and log in as user admin (the password is password by default).

  3. Execute the switchshow command to display the switches and their WWPNs. For example:

    brocade04:admin> switchshow
    switchName:       brocade04
    switchType:       2.4
    switchState:      Online
    switchRole:       Principal
    switchDomain:     6
    switchId:         fffc06 
    switchWwn:        10:00:00:60:69:12:11:9e 
    switchBeacon:     OFF
    port   0:  sw  Online       F-Port  20:00:00:01:73:00:2c:0b
    port   1:  cu  Online       F-Port  21:00:00:e0:8b:02:36:49
    port   2:  cu  Online       F-Port  21:00:00:e0:8b:02:12:49
    port   3:  sw  Online       F-Port  20:00:00:01:73:00:2d:3e
    port   4:  cu  Online       F-Port  21:00:00:e0:8b:02:18:96 
    port   5:  cu  Online       F-Port  21:00:00:e0:8b:00:90:8e
    port   6:  sw  Online       F-Port  20:00:00:01:73:00:3b:5f
    port   7:  sw  Online       F-Port  20:00:00:01:73:00:33:76
    port   8:  sw  Online       F-Port  21:00:00:e0:8b:01:d2:57
    port   9:  sw  Online       F-Port  21:00:00:e0:8b:01:0c:57
    port  10:  sw  Online       F-Port  20:08:00:a0:b8:0c:13:c9
    port  11:  sw  Online       F-Port  20:0a:00:a0:b8:0c:04:5a
    port  12:  sw  Online       F-Port  20:0c:00:a0:b8:0c:24:76
    port  13:  sw  Online       L-Port  1 public
    port  14:  sw  No_Light    
    port  15:  cu  Online       F-Port  21:00:00:e0:8b:00:42:d8

    The WWPN is the hexadecimal string to the right of the port number. For example, the WWPN for port 0 is 2000000173002c0b. (You must remove the colons from the WWPN reported in the switchshow output to produce the string to be used in the /etc/fencing.conf file.)

  4. Edit or create the /etc/fencing.conf file on the AIX node and add the WWPN for the port determined in step 1. (Comment lines begin with a # character.) For example, if you determined that port 0 is the port connected to the switch, your /etc/fencing.conf file should appear as follows:

    2000000173002c0b 

  5. To configure fencing, see the CXFS Administration Guide for SGI InfiniteStorage.

Start/Stop cxfs_client for AIX

The /usr/cxfs_cluster/bin/cxfs_cluster script will be invoked automatically during normal system startup and shutdown procedures. This script starts and stops the processes required to run CXFS.

To start up cxfs_client manually, enter the following:

aix# /usr/cxfs_cluster/bin/cxfs_cluster start

To stop cxfs_client manually, enter the following:

aix# /usr/cxfs_cluster/bin/cxfs_cluster stop

To stop and then start cxfs_client manually, enter the following:

aix# /usr/cxfs_cluster/bin/cxfs_cluster restart

Maintenance for AIX

This section contains the following:

Updating the CXFS Software for AIX

To upgrade the CXFS software on an AIX system, do the following:

  1. Make sure that no applications on the node are accessing files on a CXFS filesystem.

  2. Determine the name of the CXFS package that is installed. For example:

    aix# lslpp -L | grep cxfs
      SGIcxfs-aix5L         5.0.0.5   C     F    CXFS CLIENT for AIX

  3. Uninstall the old version by using the following command:

    installp -u packagename

    For example, given a package name of SGIcxfs-aix5L:

    aix# installp -u SGIcxfs-aix5L

  4. Obtain the CXFS update software from Supportfolio according to the directions in CXFS Administration Guide for SGI InfiniteStorage.

  5. Install the new version. See “Client Software Installation for AIX”.

Modifying the CXFS Software for AIX

You can modify the behavior of the CXFS client daemon (cxfs_client ) by placing options in the /usr/cxfs_cluster/bin/cxfs_client.options file. The available options are documented in the cxfs_client man page.


Caution: Some of the options are intended to be used internally by SGI only for testing purposes and do not represent supported configurations. Consult your SGI service representative before making any changes.


Recognizing Storage Changes for AIX

If you make changes to your storage configuration, you must rerun the HBA utilities to reprobe the storage. For more information, see the IBM HBA documentation.

GRIO on AIX

CXFS supports guaranteed-rate I/O (GRIO) version 2 on the AIX platform. Application bandwidth reservations must be explicitly released by the application before exit. If the application terminates unexpectedly or is killed, its bandwidth reservations are not automatically released and will cause a bandwidth leak. If this happens, the lost bandwidth could be recovered by rebooting the client node.

An AIX client can mount a GRIO-managed filesystem and supports application- and node-level reservations. An AIX client will interoperate with the dynamic bandwidth allocator for all I/O outside of any reservation.

For more information, see “Guaranteed-Rate I/O (GRIO) and CXFS” in Chapter 1 and the Guaranteed-Rate I/O Version 2 Guide.

XVM Failover V2 on AIX

Following is an example of the /etc/failover2.conf file on AIX:

/dev/hdisk199 affinity=1 preferred
/dev/hdisk135 affinity=1
/dev/hdisk231 affinity=2
/dev/hdisk167 affinity=2

For more information, see “XVM Failover and CXFS” in Chapter 1, the comments in the /etc/failover2.conf file, CXFS Administration Guide for SGI InfiniteStorage, and the XVM Volume Manager Administrator's Guide.

Troubleshooting for AIX

This section discusses the following:

Also see Chapter 11, “General Troubleshooting” and Appendix D, “Error Messages”.

Unable to Mount Filesystems on AIX

If cxfs_info reports that cms is up but XVM or the filesystem is in another state, then one or more mounts is still in the process of mounting or has failed to mount.

The CXFS node might not mount filesystems for the following reasons:

  • The client may not be able to see all the LUNs. This is usually caused by misconfiguration of the HBA or the SAN fabric:

    • Check that the ports on the Fibre Channel switch connected to the HBA are active. Physically look at the switch to confirm the light next to the port is green, or remotely check by using the switchShow command.

    • Check that the HBA configuration is correct.

    • Check that the HBA can see all the LUNs for the filesystems it is mounting.

    • Check that the operating system kernel can see all the LUN devices.

    • If the RAID device has more than one LUN mapped to different controllers, ensure the node has a Fibre Channel path to all relevant controllers.

  • The cxfs_client daemon may not be running. See “The cxfs_client Daemon is Not Started on AIX ”.

  • The filesystem may have an unsupported mount option. Check the cxfs_client.log for mount option errors or any other errors that are reported when attempting to mount the filesystem.

  • The cluster membership (cms), XVM, or the filesystems may not be up on the node. Execute the /usr/cxfs_cluster/bin/cxfs_info command to determine the current state of cms, XVM, and the filesystems. If the node is not up for each of these, then check the /var/tmp/cxfs_client log to see what actions have failed.

    Do the following:

    • If cms is not up, check the following:

    • If XVM is not up, check that the HBA is active and can see the LUNs.

    • If the filesystem is not up, check that one or more filesystems are configured to be mounted on this node and check the /var/tmp/cxfs_client file for mount errors.

The cxfs_client Daemon is Not Started on AIX

Confirm that the cxfs_client is not running. The following command would list the cxfs_client process if it were running:

aix# ps -ef | grep cxfs_client

The cxfs_client daemon might not start for the following reasons:

  • The workstation is in 32-bit kernel mode, which is indicated if the following message is output to the console:

    CXFS works only in the 64 bit kernel mode

    In this case, you must change to 64-bit mode as follows:

    1. Link the following libraries:

      aix# ln -fs /usr/lib/boot/unix_64 /unix
      aix# ln -fs /usr/lib/boot/unix_64 /usr/lib/boot/unix

    2. Create the boot image:

      aix# bosboot -ad /dev/ipldevice

    3. Reboot the system.

Filesystems Do Not Mount on AIX

If the /var/tmp filesystem is full, CXFS cannot write logs to it and the CXFS filesystem will not be able to mount on the AIX node. In this case, you should clean out the /var/tmp filesystem.

If a disk is read from an AIX node and the following message is output, it means that the Fibre Channel switch has broken down:

no such device or address

In this case, you should restart the Fibre Channel switch.

Panic Occurs when Executing cxfs_cluster on AIX

If the following message is output, then the genkex command does not exist:

genkex isn't found

In this case, you must install the bos.perf.tools file set.

A Memory Error Occurs with cp -p on AIX

If an error occurs when a file is copied with the cp -p command and the following message is output, there is a problem with NFS:

There is not enough memory available now

In this case, you must use maintenance level 5100-04+IY42428.

For more information, see:

https://techsupport.services.ibm.com/server/aix.fdc

An ACL Problem Occurs with cp -p on AIX

If an ACL is not reflected when a file with an ACL is copied from JFS to CXFS using the cp -p command, there is a problem with the AIX software. (The ACL information for the file is indicated by the aclget command.) In this case, you must use maintenance level 5100-04.

For more information, see:

https://techsupport.services.ibm.com/server/aix.fdc

Large Log Files on AIX

The /var/tmp/cxfs_client log file may become quite large over a period of time if the verbosity level is increased. To manually rotate this log file, use the -z option in the /usr/cxfs_cluster/bin/cxfs_client.options file.

See the cxfs_client man page and “Log Files on AIX”.

Reporting AIX Problems

When reporting a problem about a CXFS AIX node to SGI, you should retain the following information:

  • Information about the AIX node system dump and system configuration:

    aix# snap -a -o /dev/rmt0

  • Console log:

    aix# alog -o -t console

  • Current syslog file

  • The /var/tmp/cxfs_client CXFS log file

  • Moduler debugger output from the kdb command:

    • For panics or generated dumps, use the following commands and save the output:

      aix# kdb /var/adm/ras/vmcore.xx[/unix]
      (0)> stat 

    • For dumps from hangs:

      aix# kdb /var/adm/ras/vmcore.xx[/unix]
      (0)> th* (to find the slot value of the working process or thread)
      (0)> sw slot_value 
      (0)> stat

  • A list of the installed CXFS packages. Use the lslpp command as follows:

    aix# lslpp -l SGIcxfs-aix5L

  • The version information of the operating system. Use the following oslevel commands:

    aix# oslevel -r
    aix# oslevel -g | grep bos.64bit

  • A list of the loaded AIX kernel extensions. Use the genkex command.

If any of these AIX tools are not currently installed on your AIX node, you should install them.

CXFS MultiOS Client-Only Guide for SGI InfiniteStorage
(document number: 007-4507-016 / published: 2008-04-16)    table of contents  |  additional info  |  download

    Front Matter
    New Features in this Guide
    About This Guide
    Chapter 1. Introduction
    Chapter 2. Best Practices for Client-Only Nodes
    Chapter 3. AIX Platform
    Chapter 4. IRIX Platform
    Chapter 5. Linux Third-Party Platforms
    Chapter 6. Mac OS X Platform
    Chapter 7. SGI ProPack Client-Only Platform
    Chapter 8. Solaris Platform
    Chapter 9. Windows Platforms
    Chapter 10. Cluster Configuration
    Chapter 11. General Troubleshooting
    Appendix A. Operating System Path Differences
    Appendix B. Filesystem and Logical Unit Specifications
    Appendix C. Mount Options Support
    Appendix D. Error Messages
    Appendix E. cmgr Examples
    Appendix F. Summary of New Features from Previous Releases
    Glossary
    Index


home/search | what's new | help