Using the included diskfull plugin script

Change the node's or group's pre script section. In the below, we change node001's pre script section:

# luna node change -pre node001

Once the editor starts, add and write/quit:

cat << EOF >> /tmp/my-local-disk.sh
export MY_LOCAL_DISK_NAME=/dev/sda  # <--- make sure this one is correct!
export MY_LOCAL_DISK_SECTORS=   # nr of sectors, optional. if defined, then verified.
export PARTITION_MY_DISK=yes    # setting this one and the next one to 'no',
export FORMAT_MY_DISK=yes       #   will just sync with the image. can be set after first install.
export MAKE_BOOT=yes            # configures and installs grub/shim for standalone boots
EOF

Instead of /dev/sda, any disk present at boot time can be used, including devices in /dev/disk/by-id or alike.

It comes highly recommended to set the MY_LOCAL_DISK_SECTORS variable to ensure that the disk choosen is indeed the disk intended, not e.g. any other other disk with your years of valuable irreplaceable work. The sector size can be obtained through e.g. cat /sys/block/sda/queue/hw_sector_size.

Then configure the node to use the diskfull script:

# luna node change --scripts diskfull node001

Reboot the node

Using the included raid1 plugin script

change the node's or group's pre script section. In the below, we change node001's pre script section:

# luna node change -pre node001

once the editor starts, add and write/quit:

cat << EOF >> /tmp/my-local-disk.sh
export MY_LOCAL_DISK1_NAME=/dev/sda   # <--- make sure this is correct!
export MY_LOCAL_DISK2_NAME=/dev/sdb   # <--- this one too
export MY_LOCAL_DISK1_SECTORS=   # nr of sectors, optional. if defined, then verified.
export MY_LOCAL_DISK2_SECTORS=   # nr of sectors, optional. if defined, then verified.
export PARTITION_MY_DISK=yes
export FORMAT_MY_DISK=yes
export MAKE_BOOT=yes             # configures and installs grub/shim for standalone boots
EOF

Instead of /dev/sda or /dev/sdb, any disk present at boot time can be used, including devices in /dev/disk/by-id or alike

It comes highly recommended to set the MY_LOCAL_DISK{1,2}_SECTORS variables to ensure that the disk choosen is indeed the disk intended, not e.g. any other other disk with your years of valuable irreplaceable work. The sector size can be obtained through e.g. cat /sys/block/sda/queue/hw_sector_size.

Then configure the node to use the raid1 script:

# luna node change --scripts raid1 node001

Reboot the node

Disk Full node installation - inspirations

By default nodes boot diskless and utilize a part of the node's RAM to serve as root filesystem. In some scenarios it's preferable to utilize a local disk. The below scriplets provide a good start for such a way, should be modified to ones needs and are typically set in Luna on a group or node level. Also refer to luna for more detailed luna commands.

Group or node level configuration

The partitioning sticks with the group, this ensures configuration stays with the groups (and inherited nodes), when the underlying image is changed. It can however be overridden and set on a node or sets of nodes.

Partitioning in general: Disk layout (partscript)

The disk layout is two-fold, the partscript and the postscript which is responsible for the actual mounting of the configurations.

For any other configuration than diskless, the tools required for creating the filesystems and configuration are required to be present. The default list already supports most common configurations (ex4, swap, mdraid, etc.).

/trinity/images/compute/etc/dracut.conf.d/trinity.conf for reference:

install_items+=" /usr/sbin/mkswap  /usr/sbin/swapon  /usr/sbin/swaplabel  /usr/sbin/mkfs.vfat
  /usr/sbin/mkfs.fat  /usr/sbin/mkdosfs  /usr/sbin/dosfslabel  /usr/sbin/fatlabel  
  /etc/mke2fs.conf  /usr/sbin/mdadm  /usr/bin/top  /usr/sbin/fdisk  /usr/sbin/lspci
  /usr/bin/lsblk  /usr/bin/lsscsi  /usr/bin/tee  /usr/bin/lbzip2  /usr/bin/tpm2_pcrread
  /usr/bin/aria2c "

Note that disklabels can be either gpt or msdos.

Default diskless booting

The default is diskless, which does not require any disk in the node and instead the operating system will be installed into the system memory (tmpfs). This partscript also does not touch any local disk configuration which makes it ideal to dedicate the entire local disk for scratch or any other purpose.

mount -t tmpfs tmpfs /sysroot

The postscript:

 echo 'tmpfs / tmpfs defaults 0 0' >> /sysroot/etc/fstab

Diskfull setup steps

The steps below are performed on the headnode or controller.

As mentioned, the installation scripts, pre, part and post are typically configured on a group level where all members inherit the higher level settings. However, the installation scripts can also be set on a node level. This recipe will use the group level configuration while working on a cloned group.

Step 1: prescript

create a file called /tmp/part.txt with the following content:

parted /dev/sda -s 'rm 1; rm 2; rm 3'

parted /dev/sda -s 'mklabel gpt'

parted /dev/sda -s 'mkpart efi fat32 1 512m'     # EFI System Partition
parted /dev/sda -s 'mkpart boot ext2 512m 1g'     # Boot partition
parted /dev/sda -s 'mkpart root ext4 1g 100%'     # Root partition

parted /dev/sda -s 'set 1 boot on'
parted /dev/sda -s 'name 1 "EFI System Partition"'

while [[ ! -b /dev/sda3 ]]; do sleep 1; done

mkfs.fat -F32 /dev/sda1  # Format EFI partition (FAT32)
mkfs.ext2 /dev/sda2      # Format Boot partition (EXT2)
mkfs.ext4 /dev/sda3      # Format Root partition (EXT4)

mount /dev/sda3 /sysroot         # Mount root partition
mkdir -p /sysroot/boot          # Create boot directory
mount /dev/sda2 /sysroot/boot   # Mount boot partition
mkdir -p /sysroot/boot/efi      # Create EFI directory
mount /dev/sda1 /sysroot/boot/efi  # Mount EFI partition

Note the while which is required since the partitioning is very quick but the kernel might not yet pick up the new layout and thus directly fail since sda3 (in this example) might still not be activated.

Step 2: postscript

create a file called /tmp/post.txt with the following content:

mkdir /sysroot/proc /sysroot/dev /sysroot/sys
mount -t proc proc /sysroot/proc
mount -t devtmpfs devtmpfs /sysroot/dev
mount -t sysfs sysfs /sysroot/sys

cat << EOF >> /sysroot/etc/fstab
/dev/sda3   /       ext4    defaults        1 1
/dev/sda2   /boot   ext4    defaults        1 2
/dev/sda1   /boot/efi   vfat    defaults        1 2
EOF

# Remove (our) existing shim as it will not work after reinstall
# Note that this won't work with 9+ entries. i don't look for hex 10+ (a,b,c...)
SH=$(chroot /sysroot /bin/bash -c "efibootmgr -v|grep Shim1|grep -oE '^Boot[0-9]+'|grep -oE '[0-9]+'")
if [ "$SH" ]; then
        echo 'Shim found on boot '$SH
        chroot /sysroot /bin/bash -c "efibootmgr -B -b $SH"
        echo Remove
        chroot /sysroot /bin/bash -c "efibootmgr -v"
        echo Clean
fi
# install a fresh one...
chroot /sysroot /bin/bash -c "efibootmgr --verbose --disk /dev/sda --part 1 --create --label \"Shim1\" --loader /EFI/rocky/shimx64.efi"
# that we now point to Grub2
# note that we have 'rocky' in the path. this varies between distros
chroot /sysroot /bin/bash -c "grub2-mkconfig -o /boot/efi/EFI/rocky/grub.cfg"

umount /sysroot/sys
umount /sysroot/dev
umount /sysroot/proc

Step 3: configuring the group

luna group clone -qpart /tmp/part.txt -qpost /tmp/post.txt compute diskfull-compute

Step 4: moving a node to become diskfull

luna node change -g diskfull-compute node001

Node001 can now be rebooted and will become a diskfull node during installation.

Example scripts for Software RAID1

To install the operating system onto the local disk (assuming sda and sbd in this example):

Note the while which is required since the partitioning is very quick but the kernel might not yet pick up the new layout and thus directly fail since sdb3 (in this example) might still not be activated.

Note that this setup requires the installation of the MDRaid utilities (installed by default via Ansible)

partscript

DISK1=/dev/sda
DISK2=/dev/sdb

for DISK in $DISK1 $DISK2; do
    parted /dev/$DISK -s 'rm 1; rm 2; rm 3'
    parted /dev/$DISK -s 'mklabel gpt'
    parted /dev/$DISK -s 'mkpart efi fat32 1 512m'
    parted /dev/$DISK -s 'mkpart boot ext2 512m 1g'
    parted /dev/$DISK -s 'mkpart root ext4 1g 100%'
    parted /dev/$DISK -s 'set 1 boot on'
    parted /dev/$DISK -s 'set 2 raid on'
    parted /dev/$DISK -s 'set 3 raid on'
    parted /dev/$DISK -s 'name 1 "EFI System Partition"'
done
while [[ ! -b /dev/sdb3 ]]; do sleep 1; done


echo y | mdadm --create /dev/md0 --metadata 1.0 --force --assume-clean --level=mirror --raid-devices=2 /dev/${DISK1}p1 /dev/${DISK2}p1
mkfs.fat -F 16 /dev/md0
echo y | mdadm --create /dev/md1 --metadata 1.0 --force --assume-clean --level=mirror --raid-devices=2 /dev/${DISK1}p2 /dev/${DISK2}p2
mkfs.ext4 /dev/md1
echo y | mdadm --create /dev/md2 --metadata 1.2 --force --assume-clean --level=mirror --raid-devices=2 /dev/${DISK1}p3 /dev/${DISK2}p3
mkfs.ext4 /dev/md2

mount /dev/md2 /sysroot
mkdir -p  /sysroot/boot
mount /dev/md1 /sysroot/boot
mkdir -p /sysroot/boot/efi
mount /dev/md0 /sysroot/boot/efi

postscript

cat << EOF >> /sysroot/etc/fstab
/dev/md0 /boot/efi vfat defaults 0 0
/dev/md1 /boot ext4 defaults 0 0
/dev/md2 /     ext4     defaults        0 0
EOF