Skip to main content
Cluster and storage switches

Migrate from a switched configuration with direct-attached storage

Contributors netapp-jolieg

You can migrate from a switched configuration with direct-attached storage by adding two new shared switches.

Supported switches

The following switches are supported:

  • Nexus 9336C-FX2

  • Nexus 3232C

The ONTAP and NX-OS versions supported in this procedure are on the Cisco Ethernet Switches page. See Cisco Ethernet switches.

Connection Ports

The switches use the following ports to connect to nodes:

  • Nexus 9336C-FX2:

    • Ports 1- 3: Breakout mode (4x10G) Intra-Cluster Ports, int e1/1/1-4, e1/2/1-4, e1/3/1-4

    • Ports 4- 6: Breakout mode (4x25G) Intra-Cluster/HA Ports, int e1/4/1-4, e1/5/1-4, e1/6/1-4

    • Ports 7-34: 40/100GbE Intra-Cluster/HA Ports, int e1/7-34

  • Nexus 3232C:

    • Ports 1-30: 10/40/100 GbE

  • The switches use the following Inter-Switch Link (ISL) ports:

    • Ports int e1/35-36: Nexus 9336C-FX2

    • Ports e1/31-32: Nexus 3232C

The Hardware Universe contains information about supported cabling for all cluster switches.

What you'll need

  • Make sure you completed the following tasks:

    • Configured some of the ports on Nexus 9336C-FX2 switches to run at 100 GbE.

    • Planned, migrated, and documented 100 GbE connectivity from nodes to Nexus 9336C-FX2 switches.

    • Migrated nondisruptively other Cisco cluster switches from an ONTAP cluster to Cisco Nexus 9336C-FX2 network switches.

  • The existing switch network is properly set up and functioning.

  • All ports are in the up state to ensure nondisruptive operations.

  • The Nexus 9336C-FX2 switches are configured and operating under the proper version of NX-OS installed and reference configuration file (RCF) applied.

  • The existing network configuration has the following:

    • A redundant and fully functional NetApp cluster using both older Cisco switches.

    • Management connectivity and console access to both the older Cisco switches and the new switches.

    • All cluster LIFs in the up state with the cluster LIFs are on their home ports.

    • ISL ports enabled and cabled between the other Cisco switches and between the new switches.

About the examples

The examples in this procedure use the following switch and node nomenclature:

  • The existing Cisco Nexus 3232C cluster switches are c1 and c2.

  • The new Nexus 9336C-FX2 switches are sh1 and sh2.

  • The nodes are node1 and node2.

  • The cluster LIFs are node1_clus1 and node1_clus2 on node 1, and node2_clus1 and node2_clus2 on node 2 respectively.

  • Switch c2 is replaced by switch sh2 first and then switch c1 is replaced by switch sh1.

Steps

  1. If AutoSupport is enabled on this cluster, suppress automatic case creation by invoking an AutoSupport message:

    system node autosupport invoke -node * -type all -message MAINT=x h

    Where x is the duration of the maintenance window in hours.

  2. Check the administrative and operational status of each cluster port.

  3. Verify that all the cluster ports are up with a healthy status:

    network port show –role cluster

    Show example 
    cluster1::*> network port show -role cluster
Node: node1
                                                                   Ignore
                                             Speed(Mbps)  Health   Health
Port    IPspace   Broadcast Domain Link MTU  Admin/Ope    Status   Status
------- --------- ---------------- ---- ---- ------------ -------- ------
e3a     Cluster   Cluster          up   9000  auto/100000 healthy  false
e3b     Cluster   Cluster          up   9000  auto/100000 healthy  false

Node: node2
                                                                   Ignore
                                             Speed(Mbps) Health    Health
Port    IPspace   Broadcast Domain Link MTU  Admin/Oper  Status    Status
------- --------- ---------------- ---- ---- ----------- --------- ------
e3a     Cluster   Cluster          up   9000  auto/100000 healthy  false
e3b     Cluster   Cluster          up   9000  auto/100000 healthy  false
4 entries were displayed.
cluster1::*>

  1. Verify that all the cluster interfaces (LIFs) are on the home port:

    network interface show -role cluster

    Show example 
    cluster1::*> network interface show -role cluster
         Logical     Status     Network           Current     Current Is
Vserver  Interface   Admin/Oper Address/Mask      Node        Port    Home
-------  ----------- ---------- ----------------- ----------- ------- ----
Cluster
        node1_clus1  up/up      169.254.3.4/23    node1       e3a     true
        node1_clus2  up/up      169.254.3.5/23    node1       e3b     true
        node2_clus1  up/up      169.254.3.8/23    node2       e3a     true
        node2_clus2  up/up      169.254.3.9/23    node2       e3b     true
4 entries were displayed.
cluster1::*>

  1. Verify that the cluster displays information for both cluster switches:

    system cluster-switch show -is-monitoring-enabled-operational true

    Show example 
    cluster1::*> system cluster-switch show -is-monitoring-enabled-operational true
Switch                    Type               Address          Model
------------------------- ------------------ ---------------- ------
sh1                       cluster-network    10.233.205.90    N9K-C9336C
     Serial Number: FOCXXXXXXGD
      Is Monitored: true
            Reason: None
  Software Version: Cisco Nexus Operating System (NX-OS) Software, Version
                    9.3(5)
    Version Source: CDP
sh2                       cluster-network    10.233.205.91    N9K-C9336C
     Serial Number: FOCXXXXXXGS
      Is Monitored: true
            Reason: None
  Software Version: Cisco Nexus Operating System (NX-OS) Software, Version
                    9.3(5)
    Version Source: CDP
cluster1::*>

  1. Disable auto-revert on the cluster LIFs.

    Show example 
    cluster1::*> network interface modify -vserver Cluster -lif * -auto-revert false

  1. Shut down the c2 switch.

    Show example 
    c2# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
c2(config)# interface ethernet <int range>
c2(config)# shutdown

  1. Verify that the cluster LIFs have migrated to the ports hosted on cluster switch sh1:

    network interface show -role cluster

    This might take a few seconds.

    Show example 
    cluster1::*> network interface show -role cluster
          Logical     Status     Network         Current      Current  Is
Vserver   Interface   Admin/Oper Address/Mask    Node         Port     Home
--------- ----------- ---------- --------------- ------------ -------- -----
Cluster
          node1_clus1 up/up      169.254.3.4/23  node1        e3a      true
          node1_clus2 up/up      169.254.3.5/23  node1        e3a      false
          node2_clus1 up/up      169.254.3.8/23  node2        e3a      true
          node2_clus2 up/up      169.254.3.9/23  node2        e3a      false
4 entries were displayed.
cluster1::*>

  1. Replace switch c2 with the new switch sh2 and re-cable the new switch.

  2. Verify that the ports are back up on sh2. Note that the LIFs are still on switch c1.

  3. Shut down the c1 switch.

    Show example 
    c1# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
c1(config)# interface ethernet <int range>
c1(config)# shutdown

  1. Verify that the cluster LIFs have migrated to the ports hosted on cluster switch sh2. This might take a few seconds.

    Show example 
    cluster1::*> network interface show -role cluster
         Logical        Status     Network         Current   Current Is
Vserver  Interface      Admin/Oper Address/Mask    Node      Port    Home
-------- -------------- ---------- --------------- --------- ------- ----
Cluster
         node1_clus1    up/up      169.254.3.4/23  node1     e3a     true
         node1_clus2    up/up      169.254.3.5/23  node1     e3a     false
         node2_clus1    up/up      169.254.3.8/23  node2     e3a     true
         node2_clus2    up/up      169.254.3.9/23  node2     e3a     false
4 entries were displayed.
cluster1::*>

  1. Replace switch c1 with the new switch sh1 and re-cable the new switch.

  2. Verify that the ports are back up on sh1. Note that the LIFs are still on switch c2.

  3. Enable auto-revert on the cluster LIFs:

    Show example 
    cluster1::*> network interface modify -vserver Cluster -lif * -auto-revert True

  1. Verify that the cluster is healthy:

    cluster show

    Show example 
    cluster1::*> cluster show
Node                 Health  Eligibility   Epsilon
-------------------- ------- ------------- -------
node1                true    true          false
node2                true    true          false
2 entries were displayed.
cluster1::*>