Creating a Disk Full node

By default nodes boot diskless and utilize a part of the node's RAM to serve as root filesystem. In some scenarios it's preferable to utilize a local disk. The below scriplets provide a good start for such a way, should be modified to ones needs and are typically set in Luna on a group or node level. Also refer to luna for more detailed luna commands.

Group or node level configuration

The partitioning sticks with the group, this ensures configuration stays with the groups (and inherited nodes), when the underlying image is changed. It can however be overridden and set on a node or sets of nodes.

Partitioning in general: Disk layout (partscript)

The disk layout is two-fold, the partscript and the postscript which is responsible for the actual mounting of the configurations.

For any other configuration than diskless, the tools required for creating the filesystems and configuration are required to be present. The default list already supports most common configurations (ex4, swap, mdraid, etc.).

/trinity/images/compute/etc/dracut.conf.d/trinity.conf for reference:

install_items+=" /usr/sbin/mkswap  /usr/sbin/swapon  /usr/sbin/swaplabel  /usr/sbin/mkfs.vfat
  /usr/sbin/mkfs.fat  /usr/sbin/mkdosfs  /usr/sbin/dosfslabel  /usr/sbin/fatlabel  
  /etc/mke2fs.conf  /usr/sbin/mdadm  /usr/bin/top  /usr/sbin/fdisk  /usr/sbin/lspci
  /usr/bin/lsblk  /usr/bin/lsscsi  /usr/bin/tee  /usr/bin/lbzip2  /usr/bin/tpm2_pcrread
  /usr/bin/aria2c "

Note that disklabels can be either gpt or msdos.

Default diskless booting

The default is diskless, which does not require any disk in the node and instead the operating system will be installed into the system memory (tmpfs). This partscript also does not touch any local disk configuration which makes it ideal to dedicate the entire local disk for scratch or any other purpose.

mount -t tmpfs tmpfs /sysroot

The postscript:

 echo 'tmpfs / tmpfs defaults 0 0' >> /sysroot/etc/fstab

---

Diskfull setup steps

The steps below are performed on the headnode or controller.

As mentioned, the installation scripts, pre, part and post are typically configured on a group level where all members inherit the higher level settings. However, the installation scripts can also be set on a node level. This recipe will use the group level configuration while working on a cloned group.

Step 1: prescript

create a file called /tmp/part.txt with the following content:

parted /dev/sda -s 'rm 1; rm 2; rm 3'
parted /dev/sda -s 'mklabel gpt'
parted /dev/sda -s 'mkpart efi fat32 1 512m'
parted /dev/sda -s 'mkpart boot ext2 512m 1g'
parted /dev/sda -s 'mkpart root ext4 1g 100%'
parted /dev/sda -s 'set 1 boot on'
parted /dev/sda -s 'name 1 "EFI System Partition"'

while [[ ! -b /dev/sda3 ]]; do sleep 1; done

mount /dev/sda3 /sysroot
mkdir -p  /sysroot/boot
mount /dev/sda2 /sysroot/boot
mkdir -p /sysroot/boot/efi
mount /dev/sda1 /sysroot/boot/efi

Note the while which is required since the partitioning is very quick but the kernel might not yet pick up the new layout and thus directly fail since sda3 (in this example) might still not be activated.

Step 2: postscript

create a file called /tmp/post.txt with the following content:

mkdir /sysroot/proc /sysroot/dev /sysroot/sys
mount -t proc proc /sysroot/proc
mount -t devtmpfs devtmpfs /sysroot/dev
mount -t sysfs sysfs /sysroot/sys

cat << EOF >> /sysroot/etc/fstab
/dev/sda3   /       ext4    defaults        1 1
/dev/sda2   /boot   ext4    defaults        1 2
/dev/sda1   /boot/efi   vfat    defaults        1 2
EOF

# Remove (our) existing shim as it will not work after reinstall
# Note that this won't work with 9+ entries. i don't look for hex 10+ (a,b,c...)
SH=$(chroot /sysroot /bin/bash -c "efibootmgr -v|grep Shim1|grep -oE '^Boot[0-9]+'|grep -oE '[0-9]+'")
if [ "$SH" ]; then
        echo 'Shim found on boot '$SH
        chroot /sysroot /bin/bash -c "efibootmgr -B -b $SH"
        echo Remove
        chroot /sysroot /bin/bash -c "efibootmgr -v"
        echo Clean
fi
# install a fresh one...
chroot /sysroot /bin/bash -c "efibootmgr --verbose --disk /dev/sda --part 1 --create --label \"Shim1\" --loader /EFI/rocky/shimx64.efi"
# that we now point to Grub2
# note that we have 'rocky' in the path. this varies between distros
chroot /sysroot /bin/bash -c "grub2-mkconfig -o /boot/efi/EFI/rocky/grub.cfg"

umount /sysroot/sys
umount /sysroot/dev
umount /sysroot/proc

Step 3: configuring the group

luna group clone -qpart /tmp/part.txt -qpost /tmp/post.txt compute diskless-compute

Step 4: moving a node to become diskfull

luna node change -g diskless-compute node001

Node001 can now be rebooted and will become a diskfull node during installation.

---

Example scripts for Software RAID1

To install the operating system onto the local disk (assuming sda and sbd in this example):

Note the while which is required since the partitioning is very quick but the kernel might not yet pick up the new layout and thus directly fail since sdb3 (in this example) might still not be activated.

Note that this setup requires the installation of the MDRaid utilities (installed by default via Ansible)

partscript

DISK1=/dev/sda
DISK2=/dev/sdb

for DISK in $DISK1 $DISK2; do
    parted /dev/$DISK -s 'rm 1; rm 2; rm 3'
    parted /dev/$DISK -s 'mklabel gpt'
    parted /dev/$DISK -s 'mkpart efi fat32 1 512m'
    parted /dev/$DISK -s 'mkpart boot ext2 512m 1g'
    parted /dev/$DISK -s 'mkpart root ext4 1g 100%'
    parted /dev/$DISK -s 'set 1 boot on'
    parted /dev/$DISK -s 'set 2 raid on'
    parted /dev/$DISK -s 'set 3 raid on'
    parted /dev/$DISK -s 'name 1 "EFI System Partition"'
done
while [[ ! -b /dev/sdb3 ]]; do sleep 1; done


echo y | mdadm --create /dev/md0 --metadata 1.0 --force --assume-clean --level=mirror --raid-devices=2 /dev/${DISK1}p1 /dev/${DISK2}p1
mkfs.fat -F 16 /dev/md0
echo y | mdadm --create /dev/md1 --metadata 1.0 --force --assume-clean --level=mirror --raid-devices=2 /dev/${DISK1}p2 /dev/${DISK2}p2
mkfs.ext4 /dev/md1
echo y | mdadm --create /dev/md2 --metadata 1.2 --force --assume-clean --level=mirror --raid-devices=2 /dev/${DISK1}p3 /dev/${DISK2}p3
mkfs.ext4 /dev/md2

mount /dev/md2 /sysroot
mkdir -p  /sysroot/boot
mount /dev/md1 /sysroot/boot
mkdir -p /sysroot/boot/efi
mount /dev/md0 /sysroot/boot/efi

postscript

cat << EOF >> /sysroot/etc/fstab
/dev/md0 /boot/efi vfat defaults 0 0
/dev/md1 /boot ext4 defaults 0 0
/dev/md2 /     ext4     defaults        0 0
EOF