Get started
Create node configuration files for Ubuntu or Debian deployments
Suggest changes
PDFs
Node configuration files are small text files that provide the information the StorageGRID host service needs to start a node and connect it to the appropriate network and block storage resources. Node configuration files are used for virtual nodes and aren't used for appliance nodes.
Location for node configuration files
Place the configuration file for each StorageGRID node in the /etc/storagegrid/nodes directory on the host where the node will run. For example, if you plan to run one Admin Node, one Gateway Node, and one Storage Node on HostA, you must place three node configuration files in /etc/storagegrid/nodes on HostA.
You can create the configuration files directly on each host using a text editor, such as vim or nano, or you can create them elsewhere and move them to each host.
Naming of node configuration files
The names of the configuration files are significant. The format is node-name.conf, where node-name is a name you assign to the node. This name appears in the StorageGRID Installer and is used for node maintenance operations, such as node migration.
Node names must follow these rules:
-
Must be unique
-
Must start with a letter
-
Can contain the characters A through Z and a through z
-
Can contain the numbers 0 through 9
-
Can contain one or more hyphens (-)
-
Must be no more than 32 characters, not including the
.confextension
Any files in /etc/storagegrid/nodes that don't follow these naming conventions will not be parsed by the host service.
If you have a multi-site topology planned for your grid, a typical node naming scheme might be:
site-nodetype-nodenumber.conf
For example, you might use dc1-adm1.conf for the first Admin Node in Data Center 1, and dc2-sn3.conf for the third Storage Node in Data Center 2. However, you can use any scheme you like, as long as all node names follow the naming rules.
Contents of a node configuration file
A configuration file contains key/value pairs, with one key and one value per line. For each key/value pair, follow these rules:
-
The key and the value must be separated by an equal sign (
=) and optional whitespace. -
The keys can contain no spaces.
-
The values can contain embedded spaces.
-
Any leading or trailing whitespace is ignored.
The following table defines the values for all supported keys. Each key has one of the following designations:
-
Required: Required for every node or for the specified node types
-
Best practice: Optional, although recommended
-
Optional: Optional for all nodes
Admin Network keys
ADMIN_IP
| Value | Designation |
|---|---|
Grid Network IPv4 address of the primary Admin Node for the grid to which this node belongs. Use the same value you specified for GRID_NETWORK_IP for the grid node with NODE_TYPE = VM_Admin_Node and ADMIN_ROLE = Primary. If you omit this parameter, the node attempts to discover a primary Admin Node using mDNS. Note: This value is ignored, and might be prohibited, on the primary Admin Node. |
Best practice |
ADMIN_NETWORK_CONFIG
| Value | Designation |
|---|---|
DHCP, STATIC, or DISABLED |
Optional |
ADMIN_NETWORK_ESL
| Value | Designation |
|---|---|
Comma-separated list of subnets in CIDR notation to which this node should communicate using the Admin Network gateway. Example: |
Optional |
ADMIN_NETWORK_GATEWAY
| Value | Designation |
|---|---|
IPv4 address of the local Admin Network gateway for this node. Must be on the subnet defined by ADMIN_NETWORK_IP and ADMIN_NETWORK_MASK. This value is ignored for DHCP-configured networks. Examples:
|
Required if |
ADMIN_NETWORK_IP
| Value | Designation |
|---|---|
IPv4 address of this node on the Admin Network. This key is only required when ADMIN_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
Required when ADMIN_NETWORK_CONFIG = STATIC. Optional otherwise. |
ADMIN_NETWORK_MAC
| Value | Designation |
|---|---|
The MAC address for the Admin Network interface in the container. This field is optional. If omitted, a MAC address will be generated automatically. Must be 6 pairs of hexadecimal digits separated by colons. Example: |
Optional |
ADMIN_NETWORK_MASK
Value |
Designation |
IPv4 netmask for this node, on the Admin Network. Specify this key when ADMIN_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
Required if ADMIN_NETWORK_IP is specified and ADMIN_NETWORK_CONFIG = STATIC. Optional otherwise. |
ADMIN_NETWORK_MTU
Value |
Designation |
The maximum transmission unit (MTU) for this node on the Admin Network. Don't specify if ADMIN_NETWORK_CONFIG = DHCP. If specified, the value must be between 1280 and 9216. If omitted, 1500 is used. If you want to use jumbo frames, set the MTU to a value suitable for jumbo frames, such as 9000. Otherwise, keep the default value. IMPORTANT: The MTU value of the network must match the value configured on the switch port the node is connected to. Otherwise, network performance issues or packet loss might occur. Examples:
|
Optional |
ADMIN_NETWORK_TARGET
Value |
Designation |
Name of the host device that you will use for Admin Network access by the StorageGRID node. Only network interface names are supported. Typically, you use a different interface name than what was specified for GRID_NETWORK_TARGET or CLIENT_NETWORK_TARGET. Note: Don't use bond or bridge devices as the network target. Either configure a VLAN (or other virtual interface) on top of the bond device, or use a bridge and virtual Ethernet (veth) pair. Best practice:Specify a value even if this node will not initially have an Admin Network IP address. Then you can add an Admin Network IP address later, without having to reconfigure the node on the host. Examples:
|
Best practice |
ADMIN_NETWORK_TARGET_TYPE
Value |
Designation |
Interface (This is the only supported value.) |
Optional |
ADMIN_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC
Value |
Designation |
True or False Set the key to "true" to cause the StorageGRID container use the MAC address of the host host target interface on the Admin Network. Best practice: In networks where promiscuous mode would be required, use the ADMIN_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC key instead. For more details on MAC cloning: |
Best practice |
ADMIN_ROLE
Value |
Designation |
Primary or non-primary This key is only required when NODE_TYPE = VM_Admin_Node; don't specify it for other node types. |
Required when NODE_TYPE = VM_Admin_Node Optional otherwise. |
Block device keys
BLOCK_DEVICE_AUDIT_LOGS
Value |
Designation |
Path and name of the block device special file this node will use for persistent storage of audit logs. Examples:
|
Required for nodes with NODE_TYPE = VM_Admin_Node. Don't specify it for other node types. |
BLOCK_DEVICE_RANGEDB_nnn
Value |
Designation |
Path and name of the block device special file this node will use for persistent object storage. This key is only required for nodes with NODE_TYPE = VM_Storage_Node; don't specify it for other node types. Only BLOCK_DEVICE_RANGEDB_000 is required; the rest are optional. The block device specified for BLOCK_DEVICE_RANGEDB_000 must be at least 4 TB; the others can be smaller. Don't leave gaps. If you specify BLOCK_DEVICE_RANGEDB_005, you must also specify BLOCK_DEVICE_RANGEDB_004. Note: For compatibility with existing deployments, two-digit keys are supported for upgraded nodes. Examples:
|
Required: BLOCK_DEVICE_RANGEDB_000 Optional: BLOCK_DEVICE_RANGEDB_001 BLOCK_DEVICE_RANGEDB_002 BLOCK_DEVICE_RANGEDB_003 BLOCK_DEVICE_RANGEDB_004 BLOCK_DEVICE_RANGEDB_005 BLOCK_DEVICE_RANGEDB_006 BLOCK_DEVICE_RANGEDB_007 BLOCK_DEVICE_RANGEDB_008 BLOCK_DEVICE_RANGEDB_009 BLOCK_DEVICE_RANGEDB_010 BLOCK_DEVICE_RANGEDB_011 BLOCK_DEVICE_RANGEDB_012 BLOCK_DEVICE_RANGEDB_013 BLOCK_DEVICE_RANGEDB_014 BLOCK_DEVICE_RANGEDB_015 |
BLOCK_DEVICE_TABLES
Value |
Designation |
Path and name of the block device special file this node will use for persistent storage of database tables. This key is only required for nodes with NODE_TYPE = VM_Admin_Node; don't specify it for other node types. Examples:
|
Required |
BLOCK_DEVICE_VAR_LOCAL
Value |
Designation |
Path and name of the block device special file this node will use for its Examples:
|
Required |
Client Network keys
CLIENT_NETWORK_CONFIG
Value |
Designation |
DHCP, STATIC, or DISABLED |
Optional |
CLIENT_NETWORK_GATEWAY
Value |
Designation |
IPv4 address of the local Client Network gateway for this node, which must be on the subnet defined by CLIENT_NETWORK_IP and CLIENT_NETWORK_MASK. This value is ignored for DHCP-configured networks. Examples:
|
Optional |
CLIENT_NETWORK_IP
Value |
Designation |
IPv4 address of this node on the Client Network. This key is only required when CLIENT_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
Required when CLIENT_NETWORK_CONFIG = STATIC Optional otherwise. |
CLIENT_NETWORK_MAC
Value |
Designation |
The MAC address for the Client Network interface in the container. This field is optional. If omitted, a MAC address will be generated automatically. Must be 6 pairs of hexadecimal digits separated by colons. Example: |
Optional |
CLIENT_NETWORK_MASK
Value |
Designation |
IPv4 netmask for this node on the Client Network. Specify this key when CLIENT_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
Required if CLIENT_NETWORK_IP is specified and CLIENT_NETWORK_CONFIG = STATIC Optional otherwise. |
CLIENT_NETWORK_MTU
Value |
Designation |
The maximum transmission unit (MTU) for this node on the Client Network. Don't specify if CLIENT_NETWORK_CONFIG = DHCP. If specified, the value must be between 1280 and 9216. If omitted, 1500 is used. If you want to use jumbo frames, set the MTU to a value suitable for jumbo frames, such as 9000. Otherwise, keep the default value. IMPORTANT: The MTU value of the network must match the value configured on the switch port the node is connected to. Otherwise, network performance issues or packet loss might occur. Examples:
|
Optional |
CLIENT_NETWORK_TARGET
Value |
Designation |
Name of the host device that you will use for Client Network access by the StorageGRID node. Only network interface names are supported. Typically, you use a different interface name than what was specified for GRID_NETWORK_TARGET or ADMIN_NETWORK_TARGET. Note: Don't use bond or bridge devices as the network target. Either configure a VLAN (or other virtual interface) on top of the bond device, or use a bridge and virtual Ethernet (veth) pair. Best practice: Specify a value even if this node will not initially have a Client Network IP address. Then you can add a Client Network IP address later, without having to reconfigure the node on the host. Examples:
|
Best practice |
CLIENT_NETWORK_TARGET_TYPE
Value |
Designation |
Interface (This is only supported value.) |
Optional |
CLIENT_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC
Value |
Designation |
True or False Set the key to "true" to cause the StorageGRID container to use the MAC address of the host target interface on the Client Network. Best practice: In networks where promiscuous mode would be required, use the CLIENT_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC key instead. For more details on MAC cloning: |
Best practice |
Grid Network keys
GRID_NETWORK_CONFIG
Value |
Designation |
STATIC or DHCP Defaults to STATIC if not specified. |
Best practice |
GRID_NETWORK_GATEWAY
Value |
Designation |
IPv4 address of the local Grid Network gateway for this node, which must be on the subnet defined by GRID_NETWORK_IP and GRID_NETWORK_MASK. This value is ignored for DHCP-configured networks. If the Grid Network is a single subnet with no gateway, use either the standard gateway address for the subnet (X.Y.Z.1) or this node's GRID_NETWORK_IP value; either value will simplify potential future Grid Network expansions. |
Required |
GRID_NETWORK_IP
Value |
Designation |
IPv4 address of this node on the Grid Network. This key is only required when GRID_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
Required when GRID_NETWORK_CONFIG = STATIC Optional otherwise. |
GRID_NETWORK_MAC
Value |
Designation |
The MAC address for the Grid Network interface in the container. Must be 6 pairs of hexadecimal digits separated by colons. Example: |
Optional If omitted, a MAC address will be generated automatically. |
GRID_NETWORK_MASK
Value |
Designation |
IPv4 netmask for this node on the Grid Network. Specify this key when GRID_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
Required when GRID_NETWORK_IP is specified and GRID_NETWORK_CONFIG = STATIC. Optional otherwise. |
GRID_NETWORK_MTU
Value |
Designation |
The maximum transmission unit (MTU) for this node on the Grid Network. Don't specify if GRID_NETWORK_CONFIG = DHCP. If specified, the value must be between 1280 and 9216. If omitted, 1500 is used. If you want to use jumbo frames, set the MTU to a value suitable for jumbo frames, such as 9000. Otherwise, keep the default value. IMPORTANT: The MTU value of the network must match the value configured on the switch port the node is connected to. Otherwise, network performance issues or packet loss might occur. IMPORTANT: For the best network performance, all nodes should be configured with similar MTU values on their Grid Network interfaces. The Grid Network MTU mismatch alert is triggered if there is a significant difference in MTU settings for the Grid Network on individual nodes. The MTU values don't have to be the same for all network types. Examples:
|
Optional |
GRID_NETWORK_TARGET
Value |
Designation |
Name of the host device that you will use for Grid Network access by the StorageGRID node. Only network interface names are supported. Typically, you use a different interface name than what was specified for ADMIN_NETWORK_TARGET or CLIENT_NETWORK_TARGET. Note: Don't use bond or bridge devices as the network target. Either configure a VLAN (or other virtual interface) on top of the bond device, or use a bridge and virtual Ethernet (veth) pair. Examples:
|
Required |
GRID_NETWORK_TARGET_TYPE
Value |
Designation |
Interface (This is the only supported value.) |
Optional |
GRID_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC
Value |
Designation |
True or False Set the value of the key to "true" to cause the StorageGRID container to use the MAC address of the host target interface on the Grid Network. Best practice: In networks where promiscuous mode would be required, use the GRID_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC key instead. For more details on MAC cloning: |
Best practice |
Interfaces key
INTERFACE_TARGET_nnnn
Value |
Designation |
Name and optional description for an extra interface you want to add to this node. You can add multiple extra interfaces to each node. For nnnn, specify a unique number for each INTERFACE_TARGET entry you are adding. For the value, specify the name of the physical interface on the bare-metal host. Then, optionally, add a comma and provide a description of the interface, which is displayed on the VLAN interfaces page and the HA groups page. Example: If you add a trunk interface, you must configure a VLAN interface in StorageGRID. If you add an access interface, you can add the interface directly to an HA group; you don't need to configure a VLAN interface. |
Optional |
Maximum RAM key
MAXIMUM_RAM
Value |
Designation |
The maximum amount of RAM that this node is allowed to consume. If this key is omitted, the node has no memory restrictions. When setting this field for a production-level node, specify a value that is at least 24 GB and 16 to 32 GB less than the total system RAM. Note: The RAM value affects a node's actual metadata reserved space. See the description of what Metadata Reserved Space is. The format for this field is Examples:
Note: If you want to use this option, you must enable kernel support for memory cgroups. |
Optional |
Node type key
NODE_TYPE
Value |
Designation |
Type of node: VM_Admin_Node VM_Storage_Node VM_Archive_Node VM_API_Gateway |
Required |
Port remap keys
PORT_REMAP
Value |
Designation |
Remaps any port used by a node for internal grid node communications or external communications. Remapping ports is necessary if enterprise networking policies restrict one or more ports used by StorageGRID, as described in Internal grid node communications or External communications. IMPORTANT: Don't remap the ports you are planning to use to configure load balancer endpoints. Note: If only PORT_REMAP is set, the mapping that you specify is used for both inbound and outbound communications. If PORT_REMAP_INBOUND is also specified, PORT_REMAP applies only to outbound communications. The format used is: Example: |
Optional |
PORT_REMAP_INBOUND
Value |
Designation |
Remaps inbound communications to the specified port. If you specify PORT_REMAP_INBOUND but don't specify a value for PORT_REMAP, outbound communications for the port are unchanged. IMPORTANT: Don't remap the ports you are planning to use to configure load balancer endpoints. The format used is: Example: |
Optional |