Gentoo Diskless HOWTO for building an SSI image. Used in conjunction with the and Installing Gentoo as a Beowulf Head node, you might just end up with a Beowulf cluster in your hands!

This document only gets you to the point of having a system image which is bootable via NFS and gives you a functional console. Adding apps is up to you. If you want to make an X Terminal SSI, check out the references, I haven't had the time to MueCow Gentoo yet.

Contents [hide]

• 1 Reference articles
• 2 Present limitation
• 3 Getting to a basic Gentoo system
• 3.1 Stage 3 Install
• 3.2 Client kernel configuration
• 3.3 Specific required packages
  • 3.3.1 sys-fs/unionfs
  • 3.3.2 sys-apps/baselayout-1.12.0-r1
  • 3.3.3 net-fs/nfs-utils
• 4 Client side files to add/modify
• 4.1 Modify /etc/make.conf
• 4.2 /boot/stateless.sh
• 4.3 /etc/conf.d/local.stop
• 4.4 /etc/fstab
• 5 Important server-side files
• 5.1 /tftpboot/pxelinux.cfg/default
• 5.2 /tftpboot/BootProfiles
• 5.3 /etc/exports
• 5.4 runlevel scripts

Reference articles I try to keep these in order of importance/relevance.

More references are inserted in the relevant sections. HOWTO Diskless X terminal with read-only root This link is of main interest (Wilhelm Meier's web site on the subject) Especially the "Preparation of the client-root" section Gentoo Diskless Clients Installation, Configuration by Wilhelm Meier, Markus Muller,Zweibrucken Project MueKow Ubuntu's ThinClientIntegration Linux Boot Present limitation This procedure is to give you a base system INIT scripts which should and shouldn't run on boot time still need to be identified (ie: do we really need to run checkfs when booting an SSI system).

Update, use baselayout 1.12.*, it's much nicer on these processes Kyron 15:22, 1 June 2006 (EDT)

I am presently concentrating on building this environment on locally compatible ARCHes (ie: building the root for a PIII on a P4, Athlon-XP on amd64 Opteron...).

Even so, you will have to use the following command to chroot into your node's tree (if your on an amd64 and the nodes are older generation AMD procs (x86)): linux32 chroot ./bin/bash instead of chroot . /bin/bash My true interest is to use this approach for building a "dynamic" cluster. So I won't spend too much time on the SSI as a workstation aspect although I will work on "MueKow"ing Gentoo at one point ;P The approach explored here is a custom kernel for custom hardware. This implies that I won't get into the complicated, yet flexible, approach which requires and INItial Ram Disk(initrd).

We should use dev-util/cpuinfo-collection to identify and select the proper arch-optimised SSI Getting to a basic Gentoo system. Presently, the approach is to CHROOT into the environment. Conversations on the #gentoo-embedded IRC channel from freenode.net points to using a simple script modified for each environment. There are drawbacks to using such a script and this is not the approach tested/used here. Nonetheless, here is an example of such a script: File: PIII_SSI.sh

!/bin/bash ARCH=x86
ACCEPT_KEYWORDS="$ACCEPT_KEYWORDS x86 ~x86" ROOT=/tftproot/gentooP3
Where you want your filesystem to be CFLAGS="$CFLAGS -O2 -mcpu=pentium3 -pipe
-fomit-frame-pointer" CXXFLAGS="$CXXFLAGS ${CFLAGS}" LDFLAGS="$LDFLAGS
-L${ROOT}/lib -L${ROOT}/usr/lib" CHOST=i686-pc-linux-gnu 
Set the building environment if different...
CBUILD=x86_64-pc-linux-gnu
CTARGET=armeb-softfloat-linux-uclibc
CXX="armeb-softfloat-linux-uclibc-g++"
CC="armeb-softfloat-linux-uclibc-gcc" PKGDIR=${ROOT}/../pkg PKG_CONFIG_PATH=/usr/armeb-
softfloat-linux-uclibc/lib/pkgconfig/ USE="-* netboot sse sse2 blas mpi zlib
nls userlocales pbs minimal $USE" FEATURES="
$FEATURES -sandbox" export
PKG_CONFIG_PATH ARCH ACCEPT_KEYWORDS CFLAGS CXXFLAGS LDFLAGS ROOT CBUILD CHOST
CTARGET CC CXX PKGDIR USE FEATURES emerge -av $

Stage 3 Install Skip to the stage 3 section of the Gentoo Installation Handbook. We essentially follow the same procedure as a regular Gentoo install. We will use

/tftproot/$ARCH_OF_NODE/ 

as the install dir in our case (where $ARCH_OF_NODE is the actual arch specific for your node, we can have more than one of course).

Code: Quick copy/paste of a typical STAGE 3 root preparation export ARCH_OF_NODE=AthlonXP export GENTOO_INSTALLDIR=/tftproot/$ARCH_OF_NODE tar -xpjf

stage3 -*.tar.bz2 -C $GENTOO_INSTALLDIR wget

http://gentoo.osuosl.org/snapshots/portage-latest.tar.bz2 wget

http://gentoo.osuosl.org/snapshots/portage-latest.tar.bz2.md5sum

we want to keep the portage trees separated, distfiles can be shared though...

tar -xpjf portage-latest.tar.bz2 -C $GENTOO_INSTALLDIR/usr/ cp /etc/resolv.conf
$GENTOO_INSTALLDIR/etc/ mount -t proc none $GENTOO_INSTALLDIR/proc/ mount -o
bind /sys $GENTOO_INSTALLDIR/sys/ 

### I have better things to do than re-download packages ;)
### mkdir $GENTOO_INSTALLDIR/usr/portage/distfiles/ mount
--bind /usr/portage/distfiles/ $GENTOO_INSTALLDIR/usr/portage/distfiles/ 
### Don't forget to edit the $GENTOO_INSTALLDIR/etc/make.conf 
### We want the same locales on the nodes as the server: cat /etc/locales.build >
$GENTOO_INSTALLDIR/etc/locales.build 
### Chroot into the environment.. chroot $GENTOO_INSTALLDIR /bin/bash ln -sf /usr/share/zoneinfo/America/Montreal/etc/localtime source /etc/profile  

env-update emerge sync Client kernel configuration Apart from personnalised hardware support which you must select for your own hardware, the following options must be compiled into your kernel. Don't forget that you will want your network device driver(s) to be BUILT_IN (a * and not an M for module, more than one can be selected).

Linux Kernel Configuration: Device Drivers ---> Networking support ---> Networking options --->

• [*] IP : kernel level autoconfiguration
• [*] IP : DHCP support File systems ---> Network File System --->
• [*] Root file system on NFS Specific required packages Obviously, versions and Masking status may differ. sys-fs/unionfs FORGET the one in portage (as of writing it was to 1.1.4 and it is quite unstable). You are better off with CVS build available at their homepage (version 20060423-1600) coupled with a gentoo-srouces (2.6.16-gentoo-r7) kernel.

Once you have downloaded the package, I recommend you untarr it into the chroot environment and build the module from this environment. Note that you cannot just change into the unionfs source directory and type make unless your chroot environment is runnign the exact same kernel version as the system (server) your are building the image from. At the top of the Makefile you will find: # this should point to where your kernel headers are KVERS=$(shell uname -r) You will want to change the KVERS=$(shell uname -r) line with the kernel version you are using for your nodes.

Here is an example: KVERS="2.6.16-gentoo-r7" You will also need to add the following entry in the same Makefile: EXTRACFLAGS=-DUNIONFS_UNSUPPORTED Once you have set these two variables in th Makefile, you should be able to compile the unionfs module with: make ...and installing the module with: make install Since we use modprobe in our stateless.sh script to load this module at startup, depmod has to be run manually. Again, if the chroot's kernel is not exactly the same version as the server system your have to tweak the tool's argument with (for example):

depmod -v 2.6.16-gentoo-r7 And, since I really love em, here is a "one-liner" that automagically does all this: make clean

• make -j3
• make install
• depmod -v

  $(readlink /usr/src/linux | sed -e's:linux-::') -a Note: sys-fs/unionfs currently does not support kernels >2.6.18, as such you need to either patch the kernel directly using
  unionfs-2.0, or use sys-fs/aufs (available on gentoo-sunrise). Personally I
  have had better success with aufs than with unionfs-2.0-u1 during my tests
  (freenode:AxS, 2007-06-26) sys-apps/baselayout-1.12.0-r1 You will want at
  least this version of baselayout as it has better support for unionfs-based
  systems. net-fs/nfs-utils We will necessarily need these since we will be
  using NFS mounts. Latest version available: 1.0.6-r6 Latest version installed:
  1.0.6-r6 Size of downloaded files: 259 kB Homepage:
  

http://nfs.sourceforge.net/ Description: NFS client and server daemons License: GPL-2 Client side files to add/modify REMEMBER, all of this is done in the client's CHROOT environment! Modify /etc/make.conf Set the CFLAGS and USE flags as you see fit for your environment and hardware (you should have already done this if you started with a Stage1 system). Nothing is preventing you from building more than one SSI ;)

Here is an example for stripped down nodes (Beowulf cluster oriented). File: /etc/make.conf USE="-* sse sse2 blas mpi zlib nls userlocales pbs minimal unicode" CFLAGS="-O2 -mcpu=pentium3 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CXXFLAGS="${CFLAGS}" MAKEOPTS="-j4" /boot/stateless.sh You can download this version of the script here. This script is the one called instead of the usual init script (which is called at the end). This script sets up unionfs and sets per-node variables such as hostname. It's a regular BASH script, to you can do as you please with it ;). Note:Append init=/boot/stateless.sh softlevel=unionfs to your kernel command line and place this script in your node /boot directory if you haven't already done so. Don't forget to make stateless.sh executable and to add the mount directories chmod a+x /boot/stateless.sh mkdir -p /mnt/unionfs/var mkdir -p /mnt/unionfs/etc File: stateless.sh #!/bin/bash MODPROBE=/sbin/modprobe IFCONFIG=/sbin/ifconfig MYHOST=$(/sbin/dhcpcd -H; /bin/hostname)

# By default,
we'll use the DHCP assigned hostname NODE_NAME="Node" ahostname(){ if [ -z
${MYHOST} ]; then echo "DHCP didn't tell me my name. Generating my own
hostname..." MYHOST="${NODE_NAME}`$IFCONFIG eth0 | awk '/inet addr/ {print
$2}' | tr -t . ' ' | awk '{print $4}'`" else echo DHCP told me my hostname is
${MYHOST}... echo "Setting domainname to DHCP's settings" /sbin/dhcpcd -D fi
echo "STATELESS: Setting Hostname to $MYHOST" echo "HOSTNAME="$MYHOST"" >
/etc/conf.d/hostname /bin/hostname "$MYHOST" } getparams() { local
cmdline=$(dmesg | grep '^Kernel command line' | sed 's/^Kernel command
line://g') for pp in $cmdline; do echo $pp | grep '^softlevel=' >/dev/null
2>&1 if [ $? -eq 0 ]; then echo $pp | sed 's/softlevel=//g' return 0 fi done
echo "" return 1 } isset() { for p in $(getparams | tr ',' ' '); do if [ "$p"
== "$1" ]; then return 0 fi done return 1 } aunionfs() { isset unionfs if [ $?
-eq 0 ]; then echo "STATELESS: Loading module unionfs ..." $MODPROBE unionfs
while [ "$1" != "" ]; do echo "STATELESS: Mounting tmpfs $1 ..." mount -n -t
tmpfs -o defaults none /mnt/unionfs/$1 echo "STATELESS: Mounting $1 unionfs
..." mount -n -t unionfs -o dirs=/mnt/unionfs/$1=rw:/$1=ro none /$1 shift done
else echo "STATELESS: Not using unionfs as requested ..." fi } aunionfs etc
var ahostname exec /sbin/init /etc/conf.d/local.

stop Since we are stateless, we can speed up things a little bit. Note that this forces the node to ALWAYS shutdown (even if you press CTRL-ALT-DEL)! File: /etc/conf.d/local.stop

#/etc/conf.d/local.stop 
# This is a good place to unload any misc. 
# programs you started above. 
# For example, if you are using OSS and have 
#"/usr/local/bin/soundon" above, put 
# "/usr/local/bin/soundoff"

here.

/sbin/poweroff -f /etc/fstab This is obviously implementation dependant but
here is an example. Note that the dump/pass parameters here can have some
importance. File: /etc/fstab (without comments) 

# 192.168.1.1:/tftproot/AthlonXP / nfs ro,defaults,hard,intr,rsize=81 0 1
192.168.1.1:/home /home nfs rw,defaults,hard,intr,rsize=8192 0 2

# NOTE: The next line is critical for boot! proc /proc proc defaults 0 0 shm /dev/shm
tmpfs nodev,nosuid,noexec 0 0 Important server-side files
/tftpboot/pxelinux.cfg/default This file is crucial.

It's here that you define many parameters that could change the loaded SSI as well as the kernel being used. Refer to How do I Configure SYSLINUX?

File:

/tftpboot/pxelinux.cfg/default prompt 1 default ltsp timeout 50 say Press F1
for boot profiles, default is ltsp in 5 seconds... F1 BootProfiles label ltsp
kernel lts/2.6.9-ltsp-3/bzImage-2.6.9-ltsp-3 append init=/linuxrc rw
root=/dev/ram0 initrd=lts/2.6.9-ltsp-3/initrd-2.6.9-ltsp-3.gz label Gentoo
kernel AthlonXP/boot/vmlinuz append root=/dev/nfs
nfsroot=192.168.1.2:/tftproot/AthlonXP,hard,intr,rw udev gentoo=nodevfs
ip=dhcp init=/boot/stateless.sh softlevel=unionfs label Windows localboot 0
/tftpboot/BootProfiles 

This file is not essential but is quite useful in a multi-boot environment.

File: /tftpboot/BootProfiles Type ltsp for regular
LTSP bootup (default after 5 seconds) Type Gentoo for the AthlonXP environment
(GENTOO SSI boot dev env) Type Windows to play worms :)

 /etc/exports And finally, the NFS exports: File: /etc/exports /tftproot/AthlonXP
192.168.1.0/255.255.255.0(ro,sync,no_all_squash,no_root_squash) /home
192.168.1.0/255.255.255.0(rw,async,no_root_squash) /ScratchPad/NodeExport
192.168.1.0/255.255.255.0(rw,async,no_root_squash)

runlevel scripts This is not mandatory, we could just use the default runlevel. But here, in the kernel command line, we specify the "softlevel" to be unifonfs. This implies that you create the folder /etc/runlevels/unionfs and you add to this runlevel all apps you want fired up under these conditions.

For example: mkdir -p
/etc/runlevels/unionfs rc-update add local unionfs We'll be using NFS mounts:
rc-update add portmap unionfs rc-update add nfs unionfs rc-update add netmount
unionfs Some other useful services for I in acpid gmond sshd; do rc-update add
$I unionfs;