Create node configuration files for Red Hat Enterprise Linux or CentOS deployments
Suggest changes
PDFs
Node configuration files are small text files that provide the information the StorageGRID host service needs to start a node and connect it to the appropriate network and block storage resources. Node configuration files are used for virtual nodes and aren't used for appliance nodes.
Where do I put the node configuration files?
You must place the configuration file for each StorageGRID node in the /etc/storagegrid/nodes directory on the host where the node will run. For example, if you plan to run one Admin Node, one Gateway Node, and one Storage Node on HostA, you must place three node configuration files in /etc/storagegrid/nodes on HostA. You can create the configuration files directly on each host using a text editor, such as vim or nano, or you can create them elsewhere and move them to each host.
What do I name the node configuration files?
The names of the configuration files are significant. The format is node-name.conf, where node-name is a name you assign to the node. This name appears in the StorageGRID Installer and is used for node maintenance operations, such as node migration.
Node names must follow these rules:
-
Must be unique
-
Must start with a letter
-
Can contain the characters A through Z and a through z
-
Can contain the numbers 0 through 9
-
Can contain one or more hyphens (-)
-
Must be no more than 32 characters, not including the
.confextension
Any files in /etc/storagegrid/nodes that don't follow these naming conventions will not be parsed by the host service.
If you have a multi-site topology planned for your grid, a typical node naming scheme might be:
site-nodetype-nodenumber.conf
For example, you might use dc1-adm1.conf for the first Admin Node in Data Center 1, and dc2-sn3.conf for the third Storage Node in Data Center 2. However, you can use any scheme you like, as long as all node names follow the naming rules.
What is in a node configuration file?
The configuration files contain key/value pairs, with one key and one value per line. For each key/value pair, you must follow these rules:
-
The key and the value must be separated by an equal sign (
=) and optional whitespace. -
The keys can contain no spaces.
-
The values can contain embedded spaces.
-
Any leading or trailing whitespace is ignored.
Some keys are required for every node, while others are optional or only required for certain node types.
The table defines the acceptable values for all supported keys. In the middle column:
R: required
BP: best practice
O: optional
| Key | R, BP, or O? | Value |
|---|---|---|
ADMIN_IP |
BP |
Grid Network IPv4 address of the primary Admin Node for the grid to which this node belongs. Use the same value you specified for GRID_NETWORK_IP for the grid node with NODE_TYPE = VM_Admin_Node and ADMIN_ROLE = Primary. If you omit this parameter, the node attempts to discover a primary Admin Node using mDNS. Note: This value is ignored, and might be prohibited, on the primary Admin Node. |
ADMIN_NETWORK_CONFIG |
O |
DHCP, STATIC, or DISABLED |
ADMIN_NETWORK_ESL |
O |
Comma-separated list of subnets in CIDR notation to which this node should communicate using the Admin Network gateway. Example: |
ADMIN_NETWORK_GATEWAY |
O (R) |
IPv4 address of the local Admin Network gateway for this node. Must be on the subnet defined by ADMIN_NETWORK_IP and ADMIN_NETWORK_MASK. This value is ignored for DHCP-configured networks. Note: This parameter is required if ADMIN_NETWORK_ESL is specified. Examples:
|
ADMIN_NETWORK_IP |
O |
IPv4 address of this node on the Admin Network. This key is only required when ADMIN_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
ADMIN_NETWORK_MAC |
O |
The MAC address for the Admin Network interface in the container. This field is optional. If omitted, a MAC address will be generated automatically. Must be 6 pairs of hexadecimal digits separated by colons. Example: |
ADMIN_NETWORK_MASK |
O |
IPv4 netmask for this node, on the Admin Network. This key is only required when ADMIN_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
ADMIN_NETWORK_MTU |
O |
The maximum transmission unit (MTU) for this node on the Admin Network. Don't specify if ADMIN_NETWORK_CONFIG = DHCP. If specified, the value must be between 1280 and 9216. If omitted, 1500 is used. If you want to use jumbo frames, set the MTU to a value suitable for jumbo frames, such as 9000. Otherwise, keep the default value. IMPORTANT: The MTU value of the network must match the value configured on the switch port the node is connected to. Otherwise, network performance issues or packet loss might occur. Examples:
|
ADMIN_NETWORK_TARGET |
BP |
Name of the host device that you will use for Admin Network access by the StorageGRID node. Only network interface names are supported. Typically, you use a different interface name than what was specified for GRID_NETWORK_TARGET or CLIENT_NETWORK_TARGET. Note: Don't use bond or bridge devices as the network target. Either configure a VLAN (or other virtual interface) on top of the bond device, or use a bridge and virtual Ethernet (veth) pair. Best practice:Specify a value even if this node will not initially have an Admin Network IP address. Then you can add an Admin Network IP address later, without having to reconfigure the node on the host. Examples:
|
ADMIN_NETWORK_TARGET_TYPE |
O |
Interface (This is the only supported value.) |
ADMIN_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC |
BP |
True or False Set the key to "true" to cause the StorageGRID container use the MAC address of the host host target interface on the Admin Network. Best practice: In networks where promiscuous mode would be required, use the ADMIN_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC key instead. For more details on MAC cloning: |
ADMIN_ROLE |
R |
Primary or Non-Primary This key is only required when NODE_TYPE = VM_Admin_Node; don't specify it for other node types. |
BLOCK_DEVICE_AUDIT_LOGS |
R |
Path and name of the block device special file this node will use for persistent storage of audit logs. This key is only required for nodes with NODE_TYPE = VM_Admin_Node; don't specify it for other node types. Examples:
|
BLOCK_DEVICE_RANGEDB_000 BLOCK_DEVICE_RANGEDB_001 BLOCK_DEVICE_RANGEDB_002 BLOCK_DEVICE_RANGEDB_003 BLOCK_DEVICE_RANGEDB_004 BLOCK_DEVICE_RANGEDB_005 BLOCK_DEVICE_RANGEDB_006 BLOCK_DEVICE_RANGEDB_007 BLOCK_DEVICE_RANGEDB_008 BLOCK_DEVICE_RANGEDB_009 BLOCK_DEVICE_RANGEDB_010 BLOCK_DEVICE_RANGEDB_011 BLOCK_DEVICE_RANGEDB_012 BLOCK_DEVICE_RANGEDB_013 BLOCK_DEVICE_RANGEDB_014 BLOCK_DEVICE_RANGEDB_015 |
R |
Path and name of the block device special file this node will use for persistent object storage. This key is only required for nodes with NODE_TYPE = VM_Storage_Node; don't specify it for other node types. Only BLOCK_DEVICE_RANGEDB_000 is required; the rest are optional. The block device specified for BLOCK_DEVICE_RANGEDB_000 must be at least 4 TB; the others can be smaller. Don't leave gaps. If you specify BLOCK_DEVICE_RANGEDB_005, you must also specify BLOCK_DEVICE_RANGEDB_004. Note: For compatibility with existing deployments, two-digit keys are supported for upgraded nodes. Examples:
|
BLOCK_DEVICE_TABLES |
R |
Path and name of the block device special file this node will use for persistent storage of database tables. This key is only required for nodes with NODE_TYPE = VM_Admin_Node; don't specify it for other node types. Examples:
|
BLOCK_DEVICE_VAR_LOCAL |
R |
Path and name of the block device special file this node will use for its /var/local persistent storage. Examples:
|
CLIENT_NETWORK_CONFIG |
O |
DHCP, STATIC, or DISABLED |
CLIENT_NETWORK_GATEWAY |
O |
IPv4 address of the local Client Network gateway for this node, which must be on the subnet defined by CLIENT_NETWORK_IP and CLIENT_NETWORK_MASK. This value is ignored for DHCP-configured networks. Examples:
|
CLIENT_NETWORK_IP |
O |
IPv4 address of this node on the Client Network. This key is only required when CLIENT_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
CLIENT_NETWORK_MAC |
O |
The MAC address for the Client Network interface in the container. This field is optional. If omitted, a MAC address will be generated automatically. Must be 6 pairs of hexadecimal digits separated by colons. Example: |
CLIENT_NETWORK_MASK |
O |
IPv4 netmask for this node on the Client Network. This key is only required when CLIENT_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
CLIENT_NETWORK_MTU |
O |
The maximum transmission unit (MTU) for this node on the Client Network. Don't specify if CLIENT_NETWORK_CONFIG = DHCP. If specified, the value must be between 1280 and 9216. If omitted, 1500 is used. If you want to use jumbo frames, set the MTU to a value suitable for jumbo frames, such as 9000. Otherwise, keep the default value. IMPORTANT: The MTU value of the network must match the value configured on the switch port the node is connected to. Otherwise, network performance issues or packet loss might occur. Examples:
|
CLIENT_NETWORK_TARGET |
BP |
Name of the host device that you will use for Client Network access by the StorageGRID node. Only network interface names are supported. Typically, you use a different interface name than what was specified for GRID_NETWORK_TARGET or ADMIN_NETWORK_TARGET. Note: Don't use bond or bridge devices as the network target. Either configure a VLAN (or other virtual interface) on top of the bond device, or use a bridge and virtual Ethernet (veth) pair. Best practice: Specify a value even if this node will not initially have a Client Network IP address. Then you can add a Client Network IP address later, without having to reconfigure the node on the host. Examples:
|
CLIENT_NETWORK_TARGET_TYPE |
O |
Interface (This is only supported value.) |
CLIENT_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC |
BP |
True or False Set the key to "true" to cause the StorageGRID container to use the MAC address of the host target interface on the Client Network. Best practice: In networks where promiscuous mode would be required, use the CLIENT_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC key instead. For more details on MAC cloning: |
GRID_NETWORK_CONFIG |
BP |
STATIC or DHCP (Defaults to STATIC if not specified.) |
GRID_NETWORK_GATEWAY |
R |
IPv4 address of the local Grid Network gateway for this node, which must be on the subnet defined by GRID_NETWORK_IP and GRID_NETWORK_MASK. This value is ignored for DHCP-configured networks. If the Grid Network is a single subnet with no gateway, use either the standard gateway address for the subnet (X.Y.Z.1) or this node's GRID_NETWORK_IP value; either value will simplify potential future Grid Network expansions. |
GRID_NETWORK_IP |
R |
IPv4 address of this node on the Grid Network. This key is only required when GRID_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
GRID_NETWORK_MAC |
O |
The MAC address for the Grid Network interface in the container. This field is optional. If omitted, a MAC address will be generated automatically. Must be 6 pairs of hexadecimal digits separated by colons. Example: |
GRID_NETWORK_MASK |
O |
IPv4 netmask for this node on the Grid Network. This key is only required when GRID_NETWORK_CONFIG = STATIC; don't specify it for other values. Examples:
|
GRID_NETWORK_MTU |
O |
The maximum transmission unit (MTU) for this node on the Grid Network. Don't specify if GRID_NETWORK_CONFIG = DHCP. If specified, the value must be between 1280 and 9216. If omitted, 1500 is used. If you want to use jumbo frames, set the MTU to a value suitable for jumbo frames, such as 9000. Otherwise, keep the default value. IMPORTANT: The MTU value of the network must match the value configured on the switch port the node is connected to. Otherwise, network performance issues or packet loss might occur. IMPORTANT: For the best network performance, all nodes should be configured with similar MTU values on their Grid Network interfaces. The Grid Network MTU mismatch alert is triggered if there is a significant difference in MTU settings for the Grid Network on individual nodes. The MTU values don't have to be the same for all network types. Examples: 1500 8192 |
GRID_NETWORK_TARGET |
R |
Name of the host device that you will use for Grid Network access by the StorageGRID node. Only network interface names are supported. Typically, you use a different interface name than what was specified for ADMIN_NETWORK_TARGET or CLIENT_NETWORK_TARGET. Note: Don't use bond or bridge devices as the network target. Either configure a VLAN (or other virtual interface) on top of the bond device, or use a bridge and virtual Ethernet (veth) pair. Examples:
|
GRID_NETWORK_TARGET_TYPE |
O |
Interface (This is the only supported value.) |
GRID_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC |
BP |
True or False Set the value of the key to "true" to cause the StorageGRID container to use the MAC address of the host target interface on the Grid Network. Best practice: In networks where promiscuous mode would be required, use the GRID_NETWORK_TARGET_TYPE_INTERFACE_CLONE_MAC key instead. For more details on MAC cloning: |
INTERFACE_TARGET_nnnn |
O |
Name and optional description for an extra interface you want to add to this node. You can add multiple extra interfaces to each node. For nnnn, specify a unique number for each INTERFACE_TARGET entry you are adding. For the value, specify the name of the physical interface on the bare-metal host. Then, optionally, add a comma and provide a description of the interface, which is displayed on the VLAN interfaces page and the HA groups page. For example: If you add a trunk interface, you must configure a VLAN interface in StorageGRID. If you add an access interface, you can add the interface directly to an HA group; you don't need to configure a VLAN interface. |
MAXIMUM_RAM |
O |
The maximum amount of RAM that this node is allowed to consume. If this key is omitted, the node has no memory restrictions. When setting this field for a production-level node, specify a value that is at least 24 GB and 16 to 32 GB less than the total system RAM. Note: The RAM value affects a node's actual metadata reserved space. See the description of what Metadata Reserved Space is. The format for this field is Examples:
Note: If you want to use this option, you must enable kernel support for memory cgroups. |
NODE_TYPE |
R |
Type of node: VM_Admin_Node VM_Storage_Node VM_Archive_Node VM_API_Gateway |
PORT_REMAP |
O |
Remaps any port used by a node for internal grid node communications or external communications. Remapping ports is necessary if enterprise networking policies restrict one or more ports used by StorageGRID, as described in Internal grid node communications or External communications. IMPORTANT: Don't remap the ports you are planning to use to configure load balancer endpoints. Note: If only PORT_REMAP is set, the mapping that you specify is used for both inbound and outbound communications. If PORT_REMAP_INBOUND is also specified, PORT_REMAP applies only to outbound communications. The format used is: For example:
|
PORT_REMAP_INBOUND |
O |
Remaps inbound communications to the specified port. If you specify PORT_REMAP_INBOUND but don't specify a value for PORT_REMAP, outbound communications for the port are unchanged. IMPORTANT: Don't remap the ports you are planning to use to configure load balancer endpoints. The format used is: For example:
|