In addition to the items listed in “Requirements” in Chapter 1, using a Linux node to support CXFS requires the following:

For the latest information, see the CXFS Linux release notes.

---

Note: If you use I/O fencing and ipfilterd on a node, the ipfilterd configuration must allow communication between the node and the telnet port on the switch. Also see “Configure Firewalls for CXFS Use” in Chapter 2.

---

The following commands are shipped as part of the CXFS Linux package:

The cxfs_client and xvm commands are needed to include a client-only node in a CXFS cluster. The cxfs_info command reports the current status of this node in the CXFS cluster.

The rpm command output lists all software added; see “Linux Installation Procedure”.

For more information, see the man pages.

The cxfs_client command creates a /var/log/cxfs_client log file. You should monitor the /var/log/cxfs_client and /var/log/messages log files for problems. Look for a Membership delivered message to indicate that a cluster was formed.

The Linux platform uses the logrotate system utility to rotate the CXFS logs (as opposed to other multiOS platforms, which use the -z option to cxfs_client):

For information about the log files created on server-capable administration nodes, see the CXFS 5 Administration Guide for SGI InfiniteStorage.

Linux supports the CXFS mount scripts. See “CXFS Mount Scripts” in Chapter 1 and the CXFS 5 Administration Guide for SGI InfiniteStorage.

For RHEL nodes, in order for cxfs-reprobe to appropriately probe all of the targets on the SCSI bus, you must define a group of environment variables in the /etc/cluster/config/cxfs_client.options file. For more information, see “Using cxfs-reprobe on IRIX Nodes” in Chapter 4.

Note the following:

See also Appendix B, “Filesystem and Logical Unit Specifications”.

By default, DMAPI is turned off on SLES 10 and SLES 11 systems. If you want to mount CXFS filesystems on a SLES 10 or SLES 11 client-only node with the dmi mount option, you must set DMAPI_PROBE="yes" in the /etc/sysconfig/sysctl file on the node. Changes to the file will processed on the next reboot. After setting that system configuration file, you can immediately enable DMAPI by executing the following:

# sysctl -w fs.xfs.probe_dmapi=1

All CXFS files have UNIX mode bits (read, write, and execute) and optionally an access control list (ACL). For more information about POSIX ACLs, see the chmod and setfacl man pages.

The following procedure provides an overview of the steps required to add a private network to the Linux system. A private network is required for use with CXFS. See “Use a Private Network” in Chapter 2.

You may skip some steps, depending upon the starting conditions at your site. For details about any of these steps, see the Linux operating system documentation.

In order to test node connectivity by using the GUI, the root user on the node running the CXFS diagnostics must be able to access a remote shell using the rsh command (as root) on all other nodes in the cluster. (This test is not required when using cxfs_admin because it verifies the connectivity of each node as it is added to the cluster.)

There are several ways of accomplishing this, depending on the existing settings in the pluggable authentication modules (PAMs) and other security configuration files.

The following method works with default settings. Do the following on all nodes in the cluster:

After you have completed running the connectivity tests, you may wish to disable rsh on all cluster nodes.

For more information, see the Linux operating system documentation about PAM and the hosts.equiv man page.

For each private network on each Linux node in the pool, verify access with the ping command:

The CXFS software will be initially installed and configured by SGI personnel. This section provides an overview of those procedures. You can use the information in this section to verify the installation.

Table 5-1 and Table 5-2 provide examples of the differences in package extensions among the various processor classes supported by CXFS.

---

Note: The kernel package extensions vary by architecture. Ensure that you install the appropriate package for your processor architecture.

---

---

Note: Specific packages listed here are examples and may not match the released product.

---

Installing the CXFS client software for Linux requires approximately 50-200 MB of space, depending upon the packages installed at your site.

To install the required software on a Linux node, do the following:

The cxfs_utils package includes a Performance Co-Pilot (PCP) agent for monitoring CXFS heartbeat, CMS status and other statistics. If you want to use this feature, you must also install the following PCP packages:

These packages and are included with SGI Foundation Software. You can obtain the open source PCP package from ftp://oss.sgi.com/projects/pcp/download

Use the uname -r command to ensure the kernel installed above is running.

To verify that the CXFS software has been installed properly, use the rpm -qa command to display all of the installed packages. You can filter the output by searching for particular package name.

You can modify the behavior of the CXFS client daemon ( cxfs_client) by placing options in the /etc/cluster/config/cxfs_client.options file. The available options are documented in the cxfs_client man page.

---

Caution: Some of the options are intended to be used internally by SGI only for testing purposes and do not represent supported configurations. Consult your SGI service representative before making any changes.

---

To see if cxfs_client is using the options in cxfs_client.options, enter the following:

linux# ps -ax | grep cxfs_client
 3612 ?        S      0:00 /usr/cluster/bin/cxfs_client -i cxfs3-5
 3841 pts/0    S      0:00 grep cxfs_client

To be sure that cxfs_client is configured to start up on boot, view the chkconfig output, which should appear similar to the following:

linux# chkconfig --list | grep cxfs_client
cxfs_client               0:off  1:off  2:off  3:on   4:off  5:on   6:off

On Linux nodes, the cxfs-enumerate-wwns script enumerates the world wide names (WWNs) on the host that are known to CXFS. See “CXFS Mount Scripts” in Chapter 1.

The following script is run by cxfs_client when it reprobes the Fibre Channel controllers upon joining or rejoining membership:

/var/cluster/cxfs_client-scripts/cxfs-reprobe

For RHEL nodes, you can define a group of environment variables in the /etc/cluster/config/cxfs_client.options file in order for cxfs-reprobe to probe specific targets on the SCSI bus.

The script detects the presence of the SCSI and/or XSCSI layers on the system and defaults to probing whichever layers are detected. You can override this decision by setting CXFS_PROBE_SCSI and/or CXFS_PROBE_XSCSI to either 0 (to disable the probe) or 1 (to force the probe) on the appropriate bus.

When an XSCSI scan is performed, all buses are scanned by default. You can override this decision by specifying a space-separated list of buses in CXFS_PROBE_XSCSI_BUSES. (If you include space, you must enclose the list within single quotation marks.) For example:

export CXFS_PROBE_XSCSI_BUSES='/dev/xscsi/pci0001:00:03.0-1/bus /dev/xscsi/pci0002:00:01.0-2/bus'

When a SCSI scan is performed, a fixed range of buses/channels/IDs and LUNs are scanned; these ranges may need to be changed to ensure that all devices are found. The ranges can also be reduced to increase scanning speed if a smaller space is sufficient.

The following summarizes the environment variables (separate multiple values by white space and enclose withing single quotation marks):

For example, the following would only scan the first two SCSI buses:

export CXFS_PROBE_SCSI_BUSES='0 1'

The following would scan 16 LUNs on each bus, channel, and ID combination (all on one line):

export CXFS_PROBE_SCSI_LUNS='0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15'

Other options within the /etc/cluster/config/cxfs_client.options file begin with a - character. Following is an example cxfs_client.options file:

# Example cxfs_client.options file
#
-Dnormal -serror
export CXFS_PROBE_SCSI_BUSSES=1
export CXFS_PROBE_SCSI_LUNS='0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20'

---

Note: The - character or the term export must start in the first position of each line in the cxfs_client.options file; otherwise, they are ignored by the cxfs_client service.

---

The kernels provided for the Linux node have the Device File System (devfs) enabled. This can cause problems with locating system devices in some circumstances. See the devfs FAQ at the following location:

http://www.atnf.csiro.au/people/rgooch/linux/docs/devfs.html

Confirm that the cxfs_client is not running. The following command would list the cxfs_client process if it were running:

linux# ps -ax | grep cxfs_client

Check the cxfs_client log file for errors.

Restart cxfs_client as described in “Start/Stop cxfs_client for Linux” and watch the cxfs_client log file for errors.

To be sure that cxfs_client is configured to start up on boot, view the chkconfig output, which should appear similar to the following:

linux# chkconfig --list | grep cxfs_client
cxfs_client               0:off  1:off  2:off  3:on   4:off  5:on   6:off

If cxfs_info reports that cms is up but XVM or the filesystem is in another state, then one or more mounts is still in the process of mounting or has failed to mount.

The CXFS node might not mount filesystems for the following reasons:

By default, DMAPI is turned off on SLES 10 and SLES 11 systems. If you try to mount with the dmi mount option, you will see errors such as the following:

kernel: XFS: unknown mount option [dmi]."

See “Using the dmi Mount Option on a SLES 10 or SLES 11 Node”.

The /var/log/cxfs_client log file may become quite large over a period of time if the verbosity level is increased.

See the cxfs_client.options man page and “Log Files on Linux”.

The following output from chkconfig --list refers to the X Font Server, not the XFS filesystem, and has no association with CXFS:

xfs                       0:off  1:off  2:off  3:off  4:off  5:off  6:off