Creating node configuration files

Node configuration files are small text files that provide the information the StorageGRID Webscale host service needs to start the node and connect it to the appropriate network and block storage resources.

Where do I put the node configuration files?

You must place the configuration file for each StorageGRID Webscale node in the /etc/storagegrid/nodes directory on the host where the node will run. For example, if you plan to run one Admin Node, one API Gateway Node, and one Storage Node on HostA, you must place three node configuration files in /etc/storagegrid/nodes on HostA. You can create the configuration files directly on each host using a text editor such as vim or nano, or you can create them elsewhere and move them to each host.

What do I name the node configuration files?

The names of the configuration files are significant. The format is <node-name>.conf, where <node-name> is a name you assign to the node. This name appears in the StorageGRID Webscale Installer and is used for node maintenance operations, such as node migration.

Node names must follow these rules:

  • Must start with a letter
  • Can contain the characters A through Z and a through z
  • Can contain the numbers 0 through 9
  • Can contain one or more hyphens (-)
  • Must be no more than 32 characters, not including the .conf extension

Any files in /etc/storagegrid/nodes that do not follow these naming conventions will not be parsed by the host service.

If you have a multi-site topology planned for your grid, a typical node naming scheme might be:
<site>-<node-type><node-number>.conf
For example, you might use dc1-adm1.conf for the first Admin Node in Data Center 1, and dc2-sn3.conf for the third Storage Node in Data Center 2. However, you can use any scheme you like, as long as all node names follow the naming rules.

What is in a node configuration file?

The configuration files contain key/value pairs, with one key and one value per line. For each key/value pair, you must follow these rules:

  • The key and the value must be separated by an equal sign (=) and optional whitespace.
  • The keys can contain no spaces.
  • The values can contain embedded spaces.
  • Any leading or trailing whitespace is ignored.

Some keys are required for every node, while others are optional or only required for certain node types.

The table defines the acceptable values for all supported keys. The middle column indicates whether the key is Required (R), a Best Practice (BP), or Optional (O).

Key R, BP, or O? Value

ADMIN_IP

BP

Grid Network IPv4 address of the primary Admin Node for the grid to which this node belongs. Use the same value you specified for GRID_NETWORK_IP for the grid node with NODE_TYPE = VM_Admin_Node and ADMIN_ROLE = Primary. If you omit this parameter, the node attempts to discover a primary Admin Node using mDNS.

See "How grid nodes discover the primary Admin Node."
Note: This value is ignored on the primary Admin Node.

ADMIN_NETWORK_CONFIG

O

DHCP, STATIC, or DISABLED

(Defaults to DISABLED if not specified.)

ADMIN_NETWORK_ESL

O

Comma-separated list of subnets in CIDR notation to which this node should communicate via the Admin Network gateway.

Example: 172.16.0.0/21,172.17.0.0/21

ADMIN_NETWORK_GATEWAY

O

IPv4 address of the local Admin Network gateway for this node. Must be on the subnet defined by ADMIN_NETWORK_IP and ADMIN_NETWORK_MASK. This value will override the gateway value provided by a DHCP server.
Note: This parameter is required if ADMIN_NETWORK_ESL is specified. Validation requires a value if ESL is defined, regardless of content.
Examples:
  • 1.1.1.1
  • 10.224.4.81

ADMIN_NETWORK_IP

O

IPv4 address of this node on the Admin Network. This key is only required when ADMIN_NETWORK_CONFIG = STATIC; do not specify it for other values.

Examples:
  • 1.1.1.1
  • 10.224.4.81

ADMIN_NETWORK_MAC

O

The MAC address for the Admin Network interface in the container.

This field is optional. If omitted, a MAC address will be generated automatically.

Must be 6 pairs of hexadecimal digits separated by colons.

Example: b2:9c:02:c2:27:10

ADMIN_NETWORK_MASK

O

IPv4 netmask for this node, on the Admin Network. This key is only required when ADMIN_NETWORK_CONFIG = STATIC; do not specify it for other values.

Examples:
  • 255.255.255.0
  • 255.255.248.0

ADMIN_NETWORK_MTU

O

The maximum transmission unit (MTU) for this node on the Admin Network. Do not specify if ADMIN_NETWORK_CONFIG = DHCP. If specified, the value must be between 68 and 65535. If omitted, 1400 is used.

Examples:
  • 1500
  • 8192
ADMIN_NETWORK_TARGET BP

Name of the host device that you will use for Admin Network access by the StorageGRID Webscale node. Only network interface names are supported. Typically, you use a different interface name than what was specified for GRID_NETWORK_TARGET or CLIENT_NETWORK_TARGET.

Best practice: Specify a value even if this node will not initially have an Admin Network IP address. Then you can add an Admin Network IP address later, without having to reconfigure the node on the host.

Examples:
  • bond0.1002
  • ens256

ADMIN_NETWORK_TARGET_TYPE

O

Interface

(This is the only supported value.)

ADMIN_ROLE R

Primary or Non-Primary

This key is only required when NODE_TYPE = VM_Admin_Node; do not specify it for other node types.

BLOCK_DEVICE_AUDIT_LOGS R

Path and name of the block device special file this node will use for persistent storage of audit logs. This key is only required for nodes with NODE_TYPE = VM_Admin_Node; do not specify it for other node types.

Examples:
  • /dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0
  • /dev/disk/by-id/wwn-0x600a09800059d6df000060d757b475fd
  • /dev/mapper/sgws-adm1-audit-logs

BLOCK_DEVICE_RANGEDB_00

BLOCK_DEVICE_RANGEDB_01

BLOCK_DEVICE_RANGEDB_02

BLOCK_DEVICE_RANGEDB_03

BLOCK_DEVICE_RANGEDB_04

BLOCK_DEVICE_RANGEDB_05

BLOCK_DEVICE_RANGEDB_06

BLOCK_DEVICE_RANGEDB_07

BLOCK_DEVICE_RANGEDB_08

BLOCK_DEVICE_RANGEDB_09

BLOCK_DEVICE_RANGEDB_10

BLOCK_DEVICE_RANGEDB_11

BLOCK_DEVICE_RANGEDB_12

BLOCK_DEVICE_RANGEDB_13

BLOCK_DEVICE_RANGEDB_14

BLOCK_DEVICE_RANGEDB_15

R

Path and name of the block device special file this node will use for persistent object storage. This key is only required for nodes with NODE_TYPE = VM_Storage_Node; do not specify it for other node types.

Only BLOCK_DEVICE_RANGEDB_00 is required; the rest are optional. The block device specified for BLOCK_DEVICE_RANGEDB_00 must be at least 4 TB; the others can be smaller.
Note: Do not leave gaps. If you specify BLOCK_DEVICE_RANGEDB_05, you must also specify BLOCK_DEVICE_RANGEDB_04.
Examples:
  • /dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0
  • /dev/disk/by-id/wwn-0x600a09800059d6df000060d757b475fd
  • /dev/mapper/sgws-sn1-rangedb-0

BLOCK_DEVICE_TABLES

R

Path and name of the block device special file this node will use for persistent storage of database tables. This key is only required for nodes with NODE_TYPE = VM_Admin_Node; do not specify it for other node types.

Examples:
  • /dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0
  • /dev/disk/by-id/wwn-0x600a09800059d6df000060d757b475fd
  • /dev/mapper/sgws-adm1-tables

BLOCK_DEVICE_VAR_LOCAL

R

Path and name of the block device special file this node will use for its /var/local persistent storage.

Examples:
  • /dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0
  • /dev/disk/by-id/wwn-0x600a09800059d6df000060d757b475fd
  • /dev/mapper/sgws-sn1-var-local

CLIENT_NETWORK_CONFIG

O

DHCP, STATIC, or DISABLED

(Defaults to DISABLED if not specified.)

CLIENT_NETWORK_GATEWAY

O

IPv4 address of the local Client Network gateway for this node, which must be on the subnet defined by CLIENT_NETWORK_IP and CLIENT_NETWORK_MASK. This value will override the gateway value provided by a DHCP server.

Examples:
  • 1.1.1.1
  • 10.224.4.81

CLIENT_NETWORK_IP

O

IPv4 address of this node on the Client Network. This key is only required when CLIENT_NETWORK_CONFIG = STATIC; do not specify it for other values.

Examples:
  • 1.1.1.1
  • 10.224.4.81

CLIENT_NETWORK_MAC

O

The MAC address for the Client Network interface in the container.

This field is optional. If omitted, a MAC address will be generated automatically.

Must be 6 pairs of hexadecimal digits separated by colons.

Example: b2:9c:02:c2:27:20

CLIENT_NETWORK_MASK

O

IPv4 netmask for this node on the Client Network. This key is only required when CLIENT_NETWORK_CONFIG = STATIC; do not specify it for other values.

Examples:
  • 255.255.255.0
  • 255.255.248.0

CLIENT_NETWORK_MTU

O

The maximum transmission unit (MTU) for this node on the Client Network. Do not specify if CLIENT_NETWORK_CONFIG = DHCP. If specified, the value must be between 68 and 65535. If omitted, 1400 is used.

Examples:
  • 1500
  • 8192

CLIENT_NETWORK_TARGET

BP

Name of the host device that you will use for Client Network access by the StorageGRID Webscale node. Only network interface names are supported. Typically, you use a different interface name than what was specified for GRID_NETWORK_TARGET or ADMIN_NETWORK_TARGET.

Best practice: Specify a value even if this node will not initially have a Client Network IP address. Then you can add a Client Network IP address later, without having to reconfigure the node on the host.

Examples:
  • bond0.1003
  • ens423

CLIENT_NETWORK_TARGET_TYPE

O

Interface

(This is only supported value.)

GRID_NETWORK_CONFIG

O

STATIC or DHCP

(Defaults to STATIC if not specified.)

GRID_NETWORK_GATEWAY

R

IPv4 address of the local Grid Network gateway for this node, which must be on the subnet defined by GRID_NETWORK_IP and GRID_NETWORK_MASK. This value will override the gateway value provided by a DHCP server.

If the Grid Network is a single subnet with no gateway, use either the standard gateway address for the subnet (X.Y.Z.1) or this node’s GRID_NETWORK_IP value; either value will simplify potential future Grid Network expansions.

GRID_NETWORK_IP

R

IPv4 address of this node on the Grid Network. This key is only required when GRID_NETWORK_CONFIG = STATIC; do not specify it for other values.

Examples:
  • 1.1.1.1
  • 10.224.4.81

GRID_NETWORK_MAC

O

The MAC address for the Grid Network interface in the container.

This field is optional. If omitted, a MAC address will be generated automatically.

Must be 6 pairs of hexadecimal digits separated by colons.

Example: b2:9c:02:c2:27:30

GRID_NETWORK_MASK

O

IPv4 netmask for this node on the Grid Network. This key is only required when GRID_NETWORK_CONFIG = STATIC; do not specify it for other values.

Examples:
  • 255.255.255.0
  • 255.255.248.0

GRID_NETWORK_MTU

O

The maximum transmission unit (MTU) for this node on the Grid Network. Do not specify if GRID_NETWORK_CONFIG = DHCP. If specified, the value must be between 68 and 65535. If omitted, 1400 is used.

Examples:
  • 1500
  • 8192

GRID_NETWORK_TARGET

R

Name of the host device that you will use for Grid Network access by the StorageGRID Webscale node. Only network interface names are supported. Typically, you use a different interface name than what was specified for ADMIN_NETWORK_TARGET or CLIENT_NETWORK_TARGET.

Examples:
  • bond0.1001
  • ens192

GRID_NETWORK_TARGET_TYPE

O

Interface

(This is the only supported value.)

MAXIMUM_RAM

O

The maximum amount of RAM that this node is allowed to consume. If this key is omitted, the node has no memory restrictions. When setting this field for a production-level node, do not specify a value less than 24g (24 GB).

The format for this field is <number><unit>, where <unit> can be b, k, m, or g.

Examples:

24g

38654705664b

Note: If you want to use this option, you must enable kernel support for memory cgroups.

NODE_TYPE

R

Type of node:

  • VM_Admin_Node
  • VM_Storage_Node
  • VM_Archive_Node
  • VM_API_Gateway

PORT_REMAP

O

Remaps any port used by a node for internal grid node communications or external client communications. Remapping ports is necessary if enterprise networking policies restrict one or more ports used by StorageGRID Webscale, as described in "Internal grid node communications" or "External client communications."
Note: If only PORT_REMAP is set, the mapping that you specify is used for both inbound and outbound communications. If PORT_REMAP_INBOUND is also specified, PORT_REMAP applies only to outbound communications.

The format used is: <network type>/<protocol>/<default port used by grid node>/<new port>, where network type is grid, admin, or client, and protocol is tcp or udp.

For example:
PORT_REMAP = client/tcp/18082/443

PORT_REMAP_INBOUND

O

Remaps inbound communications to the specified port. If you specify PORT_REMAP_INBOUND but do not specify a value for PORT_REMAP, outbound communications for the port are unchanged.

The format used is: <network type>/<protocol:>/<remapped port >/<default port used by grid node>, where network type is grid, admin, or client, and protocol is tcp or udp.

For example:
PORT_REMAP_INBOUND = grid/tcp/3022/22