NFSv4 + CacheFS with LDAP Single-Sign-On on Ubuntu 12.04 HOWTO

Building LAMP cluster one of the key point is: how to synchronize file update among all servers for Apache? Here we have many choice, e.g. DRBD + OCFS2, iSCSI + OCFS2, CephFS, Rsync, etc; but one of the most simple solution is to use NFS for sharing master server's DocumentRoot, e.g. /home. For sure, we are not considering performance and bottleneck here.

This HOWTO will guide you though installation of NFSv4 server and client on Ubuntu 12.04. In order to make uid/gid mapping works, we will reference Single-Sign-On (SSO) setup with LDAP in previous HOWTO. We will also utilize CacheFS for improving the overall performance.

Server Requirement

In this example let's assume we have 2 servers: dev6c1 and dev6c2, with domain name "localdomain" and IP information as below:

172.24.145.25 dev6c1.localdomain dev6c1
172.24.145.26 dev6c2.localdomain dev6c2

Where dev6c1 will be the master for OpenLDAP, where dev6c2 will be slave that using nss-pam-ldapd as SSO.

Also we will assume your disk partition that serving /var/cache/fscache is mount with extended attributes, e.g.:

/dev/xvda1   /       ext4    defaults,grpquota,usrquota,user_xattr   0       0

Then you also need to make sure that the file system has extended attributes turned on, e.g.

tune2fs -o user_xattr /dev/xvda1

And now remount it with new attributes, or simply reboot the system:

mount -o remount /

Confirm LDAP SSO Setup

I will not detail the setup of LDAP SSO here again, so please refer that HOWTO for more detail. What we should double confirm is, from slave server we can correctly query LDAP uid/gid from master, e.g.

getent passwd | grep example

Should show similar result as:

example:*:9999:9999:Example user:/home/example:/bin/sh

Install NFSv4 Server and Client

On dev6c1, execute:

aptitude -y install nfs-kernel-server

On dev6c2, execute:

aptitude -y install nfs-common cachefilesd

NFSv4 Server without Kerberos

First of all create required folders on dev6c1 for export:

mkdir -p /exports
mkdir -p /exports/home

Edit /etc/fstab on dev6c1 so we can mount it after reboot

/home   /exports/home   none    bind,rw 0       0

Now we can manually mount it:

mount /exports/home

Because we are not activating NFSv4 security this time, in /etc/default/nfs-kernel-server we set:

NEED_SVCGSSD=no

Update /etc/default/nfs-common as:

NEED_GSSD=no
NEED_IDMAPD=yes

Update /etc/idmapd.conf as:

Domain = localdomain

Update /etc/exports:

/exports        *(rw,no_subtree_check,no_root_squash,sync,insecure,fsid=0)
/exports/home   *(rw,no_subtree_check,no_root_squash,sync,insecure)

And then restart NFSv4 server service with:

/etc/init.d/nfs-kernel-server restart

Double confirm exported folder with:

showmount -e localhost

And should report as:

/exports/home *
/exports      *

Well, looks good and work well.

NFSv4 Client without Kerberos

For NFSv4 client setup at dev6c2 is much simple. Again update /etc/default/nfs-common as:

NEED_GSSD=no
NEED_IDMAPD=yes

Update /etc/idmapd.conf as:

Domain = localdomain

Edit /etc/fstab on dev6c2 so we can mount it after reboot

dev6c1:/home        /home   nfs4    _netdev,defaults,auto,hard,intr,fsc       0       0

Which fsc is the key point for enable the CacheFS support. Now we can mount it with following command at dev6c2:

mount /home

Ok that's all. For sure you should also double check each NFS remote folder/file are in correct uid/gid pair; as long as above LDAP SSO already works, this part should be fine, too.

Reference

• https://help.ubuntu.com/community/NFSv4Howto
• http://www.cyberciti.biz/faq/centos-redhat-install-configure-cachefilesd...
• http://www.linux-mag.com/id/7378/