Image management
The images located in /trinity/images (default of trix_images) are file/directory based and represent how the node would look like after installation. The files and directories can be edited or chrooted into (using lchroot, recommended method).
Note that the image on the filesystem is not the one being distributed. Once the image is ready for distribution to nodes, they need to be packed. A tarball with the contents of the image is created and made ready for distribution. The provisioning system is handled by Luna and makes it possible to distribute the image via torrents, http(s) or any other method. Since the images can be quite large, the advantages for the torrent system are significant. Each node booting will also help other nodes get their image and the only limiting factor would be the network capacity of the cluster.
Image contents
The image directory is a simple file/directory based structure which is easily modified. All the files (including symlinks) placed here end up in the image and ultimately on the node.
# ls -l /trinity/images/compute
total 72
lrwxrwxrwx. 1 root root 7 Oct 11 2021 bin -> usr/bin
dr-xr-xr-x. 5 root root 4096 Oct 18 13:39 boot
drwxr-xr-x. 2 root root 4096 Oct 18 13:38 dev
drwxr-xr-x. 120 root root 12288 Oct 18 13:48 etc
drwxr-xr-x. 4 root root 4096 Oct 18 13:42 home
lrwxrwxrwx. 1 root root 7 Oct 11 2021 lib -> usr/lib
lrwxrwxrwx. 1 root root 9 Oct 11 2021 lib64 -> usr/lib64
drwxr-xr-x. 2 root root 4096 Oct 11 2021 media
drwxr-xr-x. 2 root root 4096 Oct 11 2021 mnt
drwxr-xr-x. 5 root root 4096 Oct 18 13:48 opt
dr-xr-xr-x. 2 root root 4096 Oct 18 13:36 proc
dr-xr-x---. 5 root root 4096 Oct 18 13:43 root
drwxr-xr-x. 25 root root 4096 Oct 18 13:48 run
lrwxrwxrwx. 1 root root 8 Oct 11 2021 sbin -> usr/sbin
drwxr-xr-x. 2 root root 4096 Oct 11 2021 srv
dr-xr-xr-x. 2 root root 4096 Oct 18 13:36 sys
drwxrwxrwt. 2 root root 4096 Oct 18 13:49 tmp
drwxr-xr-x. 5 root root 4096 Oct 18 13:40 trinity
drwxr-xr-x. 12 root root 4096 Oct 18 13:36 usr
drwxr-xr-x. 20 root root 4096 Oct 18 13:38 var
Creating an image
To create a new image, run the compute-redhat.yml playbook. This will create (default name compute) an image in /trinity/images.
Modifying and updating an image
The image can be easily modified, but files may end up accidentally on the controller node itself. A safer way is to do a chroot using lchroot {name}.
To get the available images, please refer to luna osimage list.
# lchroot compute
IMAGE PATH: /trinity/images/compute
chroot [root@compute /]$
In the image, commands such as dnf update can be applied, or software can be installed. Note that many software can be installed using the modules environment, which does not require modifying the images and is in general a better approach.
Once the modifications have been completed, the image needs to be packed.
Packing an image
The image which resides on the filesystem needs to be packed into a tarball and made ready to be distributed. This is done via luna osimage pack {name}
# luna osimage pack compute
osimage pack for compute queued
2023-10-19 14:44:12 :: queued pack osimage compute with queue_id 11
2023-10-19 14:44:12 :: packing osimage compute
2023-10-19 14:44:50 :: finished packing osimage compute
2023-10-19 14:44:50 :: building osimage compute
2023-10-19 14:46:20 :: finished building osimage compute
2023-10-19 14:46:20 :: creating provisioning for osimage compute
2023-10-19 14:46:25 :: created torrent provisioning for osimage compute
2023-10-19 14:46:26 :: created http provisioning for osimage compute
[========] Image compute Packed.
Cloning an image
There may be a need to branch off an image to install new software in or test updates.
In the following example a gpu is created based off the already existing compute image.
| Original | Clone | |
|---|---|---|
| Name | compute | gpu |
| Path | /trinity/images/compute | /trinity/images/gpu |
# luna osimage clone compute gpu
2023-10-19 15:40:54 :: queued clone osimage compute->gpu with queue_id 20
2023-10-19 15:40:54 :: copying osimage compute->gpu
2023-10-19 15:41:06 :: finished copying osimage
2023-10-19 15:41:06 :: packing osimage gpu
2023-10-19 15:41:50 :: finished packing osimage gpu
2023-10-19 15:41:50 :: building osimage gpu
2023-10-19 15:43:11 :: finished building osimage gpu
2023-10-19 15:43:11 :: creating provisioning for osimage gpu
2023-10-19 15:43:15 :: created torrent provisioning for osimage gpu
2023-10-19 15:43:16 :: created http provisioning for osimage gpu
[========] OS Image gpu Cloned.
Now the image is created and it can be modified and assigned to a group or node.
# luna osimage list
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| << Osimage >> |
+---+---------+------------------------------+---------------------------------------------------------+----------------------------+-------------------------+--------------+-----------+
| # | name | kernelversion | kernelfile | imagefile | path | distribution | osrelease |
+---+---------+------------------------------+---------------------------------------------------------+----------------------------+-------------------------+--------------+-----------+
| 1 | compute | 4.18.0-477.27.1.el8_8.x86_64 | compute-1697726652-vmlinuz-4.18.0-477.27.1.el8_8.x86_64 | compute-1697726690.tar.bz2 | /trinity/images/compute | redhat | None |
| 2 | gpu | 4.18.0-477.27.1.el8_8.x86_64 | gpu-1697730066-vmlinuz-4.18.0-477.27.1.el8_8.x86_64 | gpu-1697730110.tar.bz2 | /trinity/images/gpu | redhat | None |
+---+---------+------------------------------+---------------------------------------------------------+----------------------------+-------------------------+--------------+-----------+
Changing image for a group or node
The osimage configured with a group (default) or node (by override). It is possible to assign a different image than is configured in the group, making it very easy to quickly test one image without accidentally assigning the image to nodes who are coincidentally booting at the same time.
luna group show {name}:
# luna group show compute
+-------------------------------------------------------------------------------+
| Group => compute |
+---------------------+---------------------------------------------------------+
| name | compute |
| domain | cluster |
| osimage | compute |
| osimagetag | default |
[...]
[...]
| comment | None |
+---------------------+---------------------------------------------------------+
To view a node, use luna node show {name}. When the osimage name is followed by another name in parentheses, this means the osimage is configured by the group, in this example the group is also named compute.
# luna node show node001
+----------------------------------------------------------------------------------------+
| Node => node001 |
+---------------------+------------------------------------------------------------------+
| name | node001 |
| hostname | node001.cluster |
| group | compute |
| osimage | compute (compute) |
[...]
[...]
+---------------------+------------------------------------------------------------------+
To change the image compute for gpu for the group compute:
# luna group change --osimage gpu compute
Group, compute updated.
To change the image compute for gpu for node001 (override)
# luna node change --osimage gpu node001
Node, node001 updated.
The lack of parenthesis points out that it is an override
# luna node show node001
+----------------------------------------------------------------------------------------+
| Node => node001 |
+---------------------+------------------------------------------------------------------+
| name | node001 |
| hostname | node001.cluster |
| group | compute |
| osimage | gpu |
| osimagetag | default |
[...]
[...]
+---------------------+------------------------------------------------------------------+
To get the node to default back to the group configuration, change it to empty:
# luna node change --osimage "" node001
Node, node001 updated.
# luna node show node001
+----------------------------------------------------------------------------------------+
| Node => node001 |
+---------------------+------------------------------------------------------------------+
| name | node001 |
| hostname | node001.cluster |
| group | compute |
| osimage | compute (compute) |
| osimagetag | default (compute) |
[...]
[...]
+---------------------+------------------------------------------------------------------+
Tailoring an image post-installation
By default, the hostname and network interface configuration is tailored by the Luna node installer. There may be a need to do this for additional configuration files (i.e. hostnames in files), the postscript can be used for this.
Note the tailoring is done on the group and node level and this sticks with that, regardless of the image. Any files the postscript is modifying need to be present in those images as well.
See also node / group configuration