Network mounts

Luna keeps one description of the cluster's network filesystems: which directories are exported, who may reach them, and where they are mounted. It is a mounts document, stored on the cluster, on a group or on a node. Luna renders the NFS exports on the controllers, the /etc/fstab entries on the controllers and the nodes, and it creates the mountpoints with their owner, group and mode.

The installer already fills the cluster's document with the shares it sets up (see What the installer puts there). Use the commands below to look at it, to add your own filesystems, and to give a group or a node a different set.

The mounts document

The document is YAML or JSON. Here is a cluster document that exports two directories from the controller and mounts a third filesystem from an external server:

mounts:
  - path: /trinity/home
    server: controller
    source: /trinity/mounts/trinity/home
    options: nfsvers=4.2,rw,_netdev
    export:
      options: rw,sync
      clients:
        - to: cluster
          options: no_root_squash
  - path: /apps
    server: controller
    options: nfsvers=4.2,ro,_netdev
    mode: 0755
    export:
      clients:
        - to: cluster
          options: ro
  - path: /scratch
    server: fileserver01
    source: /export/scratch
    options: nfsvers=4.2,rw,_netdev,nofail

Every entry is keyed on its path, the mountpoint. A path is declared once per document.

Field	Meaning
`path`	The mountpoint. Required, absolute.
`type`	`nfs` (default), `lustre`, `beegfs`, `gpfs` (`mmfs` is accepted as the same), `panfs`, or `manual`.
`server`	The machine that serves the filesystem: `controller`, `self`, a node name, or any other host or address.
`source`	The directory or filesystem on the server. For `nfs` and `lustre` it defaults to `path`.
`options`	The mount options written into fstab. Default `defaults`.
`state`	`mounted` (default), `present` or `absent`, see below.
`owner`, `group`, `mode`	Applied to the mountpoint when Luna creates it. `mode` is octal, e.g. `0755` or `1777`.
`export`	Makes the server export this directory over NFS, see below.
`comment`	Free text, on the document, an entry, an export or a client.

The document also carries version: 1. The CLI adds it when you leave it out.

state decides what the fstab gets:

mounted: the fstab line and the mountpoint are written, and a controller mounts the entry straight away.
present: the fstab line and the mountpoint are written. Luna does not mount it on the controller.
absent: no fstab line and no mountpoint. An export on the entry is still served. This is how a share is exported without anything mounting it.

A manual entry is only a mountpoint: Luna creates the directory, with its owner, group and mode, and writes no fstab line. Use it for a filesystem that something else mounts.

The fstab device is <server>:<source> for nfs and lustre. For the other types it is <server>:<source> when source is set, and <server> alone otherwise. A Lustre filesystem is therefore written with its MGS as the server and the filesystem name as the source:

  - path: /lustre
    type: lustre
    server: 10.149.0.10@o2ib
    source: /fs1
    options: flock,_netdev

Exports

An export block on an nfs entry makes the entry's server export the directory. The exported directory is source, or path when there is no source.

clients is a list. Each client names who may mount it in to, and may add its own options.
to is a Luna network name, a hostname, a wildcard or a subnet. A Luna network name stands for the network's subnets: its IPv4 subnet, and its IPv6 subnet too on a dual-stack network. Anything else is written as given.
options on the export block itself is put in front of every client's own options.

The first entry above renders as this export on the controller:

/trinity/mounts/trinity/home \
 10.141.0.0/16(rw,sync,no_root_squash)

An export block without clients exports to nobody. Luna logs that it renders nothing.

`controller` and `self`

Two server names are reserved:

controller is the address the whole cluster reaches the controllers on. On an HA cluster that is the floating address, so the mount follows a failover.
self is the controller that does the rendering: the controller's own hostname in its own fstab, and on a node the controller that renders its installation.

Both can carry an export block, and each controller then exports the directory. An entry without a server is treated as a share of the controller.

A node can serve too: name it as the server, and the export is written into that node's image when it installs.

Which document a node uses

The cluster's document is the default. A group can have its own, and so can a node. The most specific document wins outright: a node with its own document uses only that one, and a group with its own replaces the cluster's for its nodes. Documents do not stack. That is unlike profiles, where a node gets its group's profiles and its own.

showmounts shows the document in force and where it comes from:

# luna cluster showmounts
# luna group showmounts compute
# luna node showmounts node001

+--------------------------------------------------------------------------------------------------------------------------------------+
|                                                  Node node001 Mounts [from cluster]                                                  |
+---------------+------+--------------+------------------------------+-------------------------------+-------+-------------------------+
|      Path     | Type |    Server    |            Source            |            Options            | State |       Exported to       |
+---------------+------+--------------+------------------------------+-------------------------------+-------+-------------------------+
| /trinity/home | nfs  |  controller  | /trinity/mounts/trinity/home |     nfsvers=4.2,rw,_netdev    |   -   | cluster(no_root_squash) |
|     /apps     | nfs  |  controller  |              -               |     nfsvers=4.2,ro,_netdev    |   -   |       cluster(ro)       |
|    /scratch   | nfs  | fileserver01 |       /export/scratch        | nfsvers=4.2,rw,_netdev,nofail |   -   |            -            |
+---------------+------+--------------+------------------------------+-------------------------------+-------+-------------------------+

Exported to lists each client with its own options; -R prints the whole document as JSON. luna cluster show, luna group show and luna node show list the paths on one line (3 mounts: /trinity/home, /apps, /scratch). On a group or a node, mounts * marks a document of its own.

Changing the document

One entry at a time

addmount adds an entry, or updates the entry at the same path:

# luna cluster addmount -qmnt "{path: /scratch, server: fileserver01, source: /export/scratch, options: 'rw,_netdev,nofail'}"
# luna group addmount compute -qmnt /root/scratch-entry.yaml
# luna node addmount node001 -qmnt '{"path": "/localscratch", "type": "manual", "mode": "1777"}'

The entry comes in-line or from a file, as YAML or JSON. When an entry already exists at that path, only the fields you give change and the others stay, so you can build an entry up in steps:

# luna node addmount node001 -qmnt "{path: /scratch, options: 'ro,_netdev'}"
Mount /scratch added to node node001.

removemount takes an entry away by its path:

# luna cluster removemount /scratch
# luna node removemount node001 /scratch

A group or a node without its own document starts from the one it uses. The first addmount or removemount on it copies that document first, and from then on the group or node has its own document:

# luna node addmount node001 -qmnt "{path: /scratch, server: fileserver01, source: /export/scratch, options: 'rw,_netdev,nofail'}"
Mount /scratch added to node node001. The cluster mounts document was copied to node node001 first; it now deviates from it.

Later changes to the cluster's document do not reach that node any more. To make it use the cluster's or the group's document again, clear its own:

# luna node change node001 -qmnt ""

The whole document

--quick-mounts (-qmnt) on luna cluster change sets the whole document at once, from a file or in-line, YAML or JSON. For groups and nodes the same option is on add, change and clone. --mounts (-mnt) opens the current document in $EDITOR as YAML:

# luna cluster change -qmnt /root/cluster-mounts.yaml
# luna group change compute --mounts

A document that does not parse is refused by the CLI, and nothing is stored.

What Luna refuses

Luna checks each document before it stores it, and says why it refuses one:

# luna node addmount node001 -qmnt "{path: /data, type: cephfs}"
Invalid request: config_validation: mounts[5].type unsupported: 'cephfs' (allowed: beegfs, gpfs, lustre, manual, mmfs, nfs, panfs).
# luna node addmount node001 -qmnt "{path: data}"
Invalid request: config_validation: mounts[5].path must be absolute: data.
# luna node removemount node001 /nothere
Invalid request: no mount at /nothere in the mounts document node node001 sees.

It also refuses:

An unknown field, anywhere in the document.
An export on an entry that is not nfs, or on one without a server.
An export whose server is neither controller, self, a controller nor a node. Nothing could serve it.
Two entries that export the same directory from the same server.
A document that exports a directory another document already exports from the same server with different export settings.

Whether a mount option is valid, or whether the server can be reached, only the machine that mounts it can tell. That shows up when the filesystem is mounted, not when it is stored.

What Luna renders

On the controllers

Every change to a mounts document is applied on every controller straight away, and again each time the Luna daemon starts:

Exports: every entry, from any document, that the controller serves (controller, self, or the controller's hostname) is written to /etc/exports.d/luna.exports, and the export table is reloaded. Do not edit that file, because Luna overwrites it.
fstab: the entries of the cluster document are written to /etc/fstab between # BEGIN luna mounts and # END luna mounts, and the mounted ones are mounted. A controller does not mount a directory it serves itself, unless source names a different directory than path.
Mountpoints and exported directories are created, with the owner, group and mode the entry gives.

Luna writes only between its markers. When /etc/fstab already has a line for the same mountpoint outside the block, Luna leaves that line in place, writes none of its own for that path, and logs it. To let Luna manage the mount, remove the other line.

A directory that is itself on a network filesystem on the controller is not exported. An NFS server cannot re-export a network mount. Name the directory behind it as the source instead.

On the nodes

A node gets its mounts when it installs, from the document it uses. The installer:

creates the mountpoints with their owner, group and mode,
writes the fstab block between the same markers, leaving any line the image already has for a mountpoint,
writes /etc/exports.d/luna.exports when the node serves an export itself.

This shows as the install.mounts step in the node's install status. A change to a document reaches a node at its next installation. A running node is not changed.

What the installer puts there

The TrinityX installer hands the shares it sets up to Luna as entries in the cluster's document: one entry per share the controllers export, including the paths from shared filesystem disks, and one per network filesystem the cluster mounts. A directory that only the controllers exchange, such as /trinity/local/sync, is exported with state: absent, so no node ever mounts it.

Those entries are ordinary entries: look at them with luna cluster showmounts and change them the same way as your own.