Configure subnet manager

Contributors netapp-driley netapp-jolieg NetAppZacharyWambold netapp-jsnyder

A subnet manager must be running in your environment on your switch or on your hosts. If you are running it host-side, use the following procedure to set it up.

Note Before configuring the subnet manager, you must install the infiniband-diags package to obtain the globally unique ID (GUID) through the ibstat -p command. See Determine host port GUIDs and make the recommended settings for information on how to install the infiniband-diags package.
Steps
  1. Install the opensm package on any hosts that will be running the subnet manager.

  2. Use the ibstat -p command to find GUID0 and GUID1 of the HCA ports. For example:

    # ibstat -p
    0x248a070300a80a80
    0x248a070300a80a81
  3. Create a subnet manager script that runs once as part of the boot process.

    # vim /usr/sbin/subnet-manager.sh
  4. Add the following lines. Substitute the values you found in step 2 for GUID0 and GUID1. For P0 and P1, use the subnet manager priorities, with 1 being the lowest and 15 the highest.

    #!/bin/bash
    
    opensm -B -g <GUID0> -p <P0> -f /var/log/opensm-ib0.log
    opensm -B -g <GUID1> -p <P1> -f /var/log/opensm-ib1.log

    An example of the command with value substitutions:

    #!/bin/bash
    
    opensm -B -g 0x248a070300a80a80 -p 15 -f /var/log/opensm-ib0.log
    opensm -B -g 0x248a070300a80a81 -p 1 -f /var/log/opensm-ib1.log
  5. Create a systemd service unit file named subnet-manager.service.

    # vim /etc/systemd/system/subnet-manager.service
  6. Add the following lines.

    [Unit]
    Description=systemd service unit file for subnet manager
    
    [Service]
    Type=forking
    ExecStart=/bin/bash /usr/sbin/subnet-manager.sh
    
    [Install]
    WantedBy=multi-user.target
  7. Notify systemd of the new service.

    # systemctl daemon-reload
  8. Enable and start the subnet-manager service.

    # systemctl enable subnet-manager.service
    # systemctl start subnet-manager.service