How to install and run GFS.

Refer to the cluster project page for the latest information.
http://sources.redhat.com/cluster/


Install
-------

Install the version of the linux kernel and openais that are compatible
with the given cluster release.  gfs requires this additional kernel
patch that exports some symbols from gfs2:
  ftp://sources.redhat.com/pub/cluster/releases/lockproto-exports.patch

Install the cluster release
cd cluster
./configure --kernel_src=/usr/src/linux
(./configure --help for more options, --libdir=/usr/lib64 needed on x86-64)
make install


Load kernel modules
-------------------

gfs2, dlm, lock_dlm, lock_nolock (from the kernel)
gfs (from cluster/gfs-kernel)


Configuration
-------------

Create /etc/cluster/cluster.conf and copy it to all nodes.

  The format and content of cluster.conf has changed little since the
  last generation of the software.  See old example here:
  http://sources.redhat.com/cluster/doc/usage.txt
  The one change you will need to make is to add nodeids for all nodes
  in the cluster. These are now mandatory. eg:

     <clusternode name="node12.mycluster.mycompany.com" nodeid="12">

  If you already have a cluster.conf file with no nodeids in it, then you can
  use the 'ccs_tool addnodeids' command to add them.


Example cluster.conf
--------------------

This is a basic cluster.conf file that requires manual fencing.  The node
names should resolve to the address on the network interface you want to
use for openais/cman/dlm communication.

<?xml version="1.0"?>
<cluster name="alpha" config_version="1">

<clusternodes>
<clusternode name="node01.mycluster.mycompany.com" nodeid="1">
        <fence>
        </fence>
</clusternode>

<clusternode name="node02.mycluster.mycompany.com" nodeid="2">
        <fence>
        </fence>
</clusternode>

<clusternode name="node03.mycluster.mycompany.com" nodeid="3">
        <fence>
        </fence>
</clusternode>
</clusternodes>

<fencedevices>
</fencedevices>

</cluster>


Startup procedure
-----------------

To use the init script, run "service cman start" on all nodes.  When getting a
cluster set up initially, it can be helpful to do all the steps manually as
follows.

Run these commands on each cluster node:

> mount -t configfs none /sys/kernel/config
> ccsd
> cman_tool join
> groupd
> fenced
> fence_tool join
> dlm_controld
> gfs_controld
> clvmd (only necessary if using clvm volumes for gfs)
> mkfs -t gfs -p lock_dlm -t <clustername>:<fsname> -j <#journals> <blockdev>
> mount -t gfs [-v] <blockdev> <mountpoint>

Notes:
- replace "gfs" with "gfs2" above to use gfs2 instead of gfs
- <clustername> in mkfs should match the one in cluster.conf.
- <fsname> in mkfs is any name you pick, each fs must have a different name.
- <#journals> in mkfs should be greater than or equal to the number of nodes
  that you want to mount this fs, each node uses a separate journal.
- To avoid unnecessary fencing when starting the cluster, it's best for
  all nodes to join the cluster (complete cman_tool join) before any
  of them do fence_tool join.
- The cman_tool "status" and "nodes" options show the status and members
  of the cluster.
- The group_tool command shows the status of fencing, dlm and gfs groups
  that the local node is part of.
- See the following man pages for more details:
  ccs(7), ccsd(8), cluster.conf(5), cman(5), cman_tool(8), groupd(8),
  group_tool(8), fence(8), fenced(8), fence_tool(8), dlm_controld(8),
  gfs_controld(8), gfs(8), gfs_mkfs(8)


Shutdown procedure
------------------

Run these commands on each cluster node:

> umount [-v] <mountpoint>
> fence_tool leave
> cman_tool leave


Converting from GFS1 to GFS2
----------------------------

If you have GFS1 filesystems that you need to convert to GFS2, follow
this procedure:

1. Back up your entire filesystem first.
   e.g. cp /dev/your_vg/lvol0 /your_gfs_backup

2. Run fsck to ensure filesystem integrity.
   e.g. gfs2_fsck /dev/your_vg/lvol0

3. Make sure the filesystem is not mounted from any node.
   e.g. for i in `grep "<clusternode name" /etc/cluster/cluster.conf | cut -d '"' -f2` ; do ssh $i "mount | grep gfs" ; done

4. Make sure you have the latest software versions.

5. Run gfs2_convert <blockdev> from one of the nodes.
   e.g. gfs2_convert /dev/your_vg/lvol0

