Installation notes

How the playbooks are meant to be used

The TrinityX playbooks install and configure a complete cluster: the controllers, their services and the node images. After the installation they can be run again: as a whole, or for the parts that need it using --tags (see the tags reference). Which of the two fits depends on how the cluster is managed, as explained below. This is how Ansible is designed: a playbook describes the desired state of a system, and every run brings the system back to that state (Getting started with Ansible, Tags, Idempotency). A run therefore resets what the roles define, including changes made by hand. The same holds for node images, see Modifying and updating an image.

A cluster is managed in one of two ways:

  • The playbooks are the source of truth. Changes are made in the playbook configuration (group_vars/all.yml and role variables), so that every run reproduces them.
  • The live configuration is the source of truth. Changes are made on the system itself, and the playbooks are only run with the tags of the components that need an update.

From TrinityX 16.1 the playbooks lean towards the second way: once they exist, a number of files, data and pacemaker resources are left as they are on a re-run (see below). Applying a changed TrinityX default to an existing cluster therefore needs -e override=yes for the components concerned.

What a re-run leaves as it is

A re-run of the playbook leaves the following as it is once it exists, so changes made to them after the installation are kept. Changes that a later TrinityX release makes to these items are therefore not applied to an existing installation; run the playbook with -e override=yes to apply them, which replaces your own changes.

  • /etc/ssh/sshd_config and /etc/corosync/corosync.conf on the controllers
  • /etc/sssd/sssd.conf, on the controllers and in every image that was set up before
  • /etc/ood/config/ood_portal.yml, on the controllers and in every login image that was set up before
  • the Luna database
  • the OpenLDAP or 389 Directory Server data
  • the Slurm configuration
  • the certificates
  • the Open OnDemand app files: app and module files, the lchroot and ibanalyzer sudoers files and the lchroot wrapper; the apps' config/luna.ini changes only with a new TrinityX OOD apps build (a run with --tags admin-change rewrites the sudoers files and the lchroot wrapper, and sets the group of the apps' config/luna.ini)
  • the pacemaker resources, cluster properties and resource defaults of an HA pair; resources that a new release adds are still created. The iSCSI and direct-disk shared-storage resources are the exception: a re-run applies them again
  • the shared disk: once created, it is not created again. A DRBD shared disk is set up again by each run until one run has completed it, pacemaker resources included

A re-run says what it left as it is: each item shows as a red failed line followed by ...ignoring, naming what was left as it is. The run continues, and the play recap counts these lines as ignored.

A kept file also keeps its current owner and permissions; a re-run does not reset them either.

To restore the items above to the TrinityX defaults, run the playbook with -e override=yes. Combine it with --tags=xyz, where xyz is one of the tags in the controller.yml playbook (see the tags reference), to limit it to one component. Run overrides with great caution: an override resets and recreates the configuration and data of the components it reaches. On an HA pair with a DRBD shared disk, it re-creates the DRBD disks and the file systems on them, and the data on the shared disk is lost. -e override=no has the same effect as leaving the option out, except for the shared disk: there any value of override, no included, re-creates the DRBD disks. Leave the option out instead.


Open OnDemand portal notes

Very Important Note

Before installation, it comes super extremely highly recommended to have knowledge on how the Open Ondemand portal will be reached. This is true for the portal hosted through the controller, as well as for dedicated login servers/images.

Please see detailed OOD information

Incorrect or incomplete information will increase the chance of a non-functioning portal


HA

shared_fs_disk

The shared_fs_disk layout should provide for {{ trix_ha }}, {{ trix_home }} and {{ trix_shared }} as an absolute minimum. A mix for these definitions may exists, thinking of e.g. {{ trix_home }} or {{ trix_shared }} being served through different means e.g. NFS or DAS, but they have to be defined or present.

The luna (os)images should typically NOT reside on a shared disk. Luna has by default a native method to synchronize images, which reside on a local disk, partition or directory.

Refer to HA or High Availability paragraph 'shared disk' or visit the extensive documentation regarding shared disks for more details.

/trinity/local, /trinity/images and /trinity/ha

/trinity/local (default of trix_local), the luna (os)images directory /trinity/images (default of trix_images) and the HA directory /trinity/ha (default of trix_ha) cannot reside on an NFS-imported share, nor on a shared disk. They must live on local block storage (a local disk, partition or directory) on the controller. /trinity/local holds per-machine configuration, is hard-configured to that path, and should never be shared between controllers or nodes. Luna has by default a native method to synchronize images, which reside on a local disk, partition or directory.


Upgrade single controller to HA

It is NOT possible to upgrade a single controller setup into an HA setup! Please make sure to select the right path going forward.