Skip to main content
ONTAP MetroCluster

Configuring Cisco IP switches

Contributors netapp-folivia netapp-aoife netapp-thomi

You must configure the Cisco IP switches for use as the cluster interconnect and for backend MetroCluster IP connectivity.

About this task

Several of the procedures in this section are independent procedures and you only need to execute those you are directed to or are relevant to your task.

Resetting the Cisco IP switch to factory defaults

Before installing any RCF file, you must erase the Cisco switch configuration and perform basic configuration. This procedure is required when you want to reinstall the same RCF file after a previous installation failed, or if you want to install a new version of an RCF file.

About this task
  • You must repeat these steps on each of the IP switches in the MetroCluster IP configuration.

  • You must be connected to the switch using the serial console.

  • This task resets the configuration of the management network.

Steps
  1. Reset the switch to factory defaults:

    1. Erase the existing configuration:

      write erase

    2. Reload the switch software:

      reload

      The system reboots and enters the configuration wizard. During the boot, if you receive the prompt “Abort Auto Provisioning and continue with normal setup? (yes/no)[n]”, you should respond yes to proceed.

    3. In the configuration wizard, enter the basic switch settings:

      • Admin password

      • Switch name

      • Out-of-band management configuration

      • Default gateway

      • SSH service (RSA)

        After completing the configuration wizard, the switch reboots.

    4. When prompted, enter the user name and password to log in to the switch.

      The following example shows the prompts and system responses when configuring the switch. The angle brackets (<<<) show where you enter the information.

      ---- System Admin Account Setup ----
      Do you want to enforce secure password standard (yes/no) [y]:y  **<<<**
      
          Enter the password for "admin": password
        Confirm the password for "admin": password
               ---- Basic System Configuration Dialog VDC: 1 ----
      
      This setup utility will guide you through the basic configuration of
      the system. Setup configures only enough connectivity for management
      of the system.
      
      Please register Cisco Nexus3000 Family devices promptly with your
      supplier. Failure to register may affect response times for initial
      service calls. Nexus3000 devices must be registered to receive
      entitled support services.
      
      Press Enter at anytime to skip a dialog. Use ctrl-c at anytime
      to skip the remaining dialogs.

      You enter basic information in the next set of prompts, including the switch name, management address, and gateway, and select SSH with RSA.

      Would you like to enter the basic configuration dialog (yes/no): yes
        Create another login account (yes/no) [n]:
        Configure read-only SNMP community string (yes/no) [n]:
        Configure read-write SNMP community string (yes/no) [n]:
        Enter the switch name : switch-name **<<<**
        Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]:
          Mgmt0 IPv4 address : management-IP-address  **<<<**
          Mgmt0 IPv4 netmask : management-IP-netmask  **<<<**
        Configure the default gateway? (yes/no) [y]: y **<<<**
          IPv4 address of the default gateway : gateway-IP-address  **<<<**
        Configure advanced IP options? (yes/no) [n]:
        Enable the telnet service? (yes/no) [n]:
        Enable the ssh service? (yes/no) [y]: y  **<<<**
          Type of ssh key you would like to generate (dsa/rsa) [rsa]: rsa **<<<**
          Number of rsa key bits <1024-2048> [1024]:
        Configure the ntp server? (yes/no) [n]:
        Configure default interface layer (L3/L2) [L2]:
        Configure default switchport interface state (shut/noshut) [noshut]: shut **<<<**
        Configure CoPP system profile (strict/moderate/lenient/dense) [strict]:

      The final set of prompts completes the configuration:

      The following configuration will be applied:
        password strength-check
        switchname IP_switch_A_1
      vrf context management
      ip route 0.0.0.0/0 10.10.99.1
      exit
        no feature telnet
        ssh key rsa 1024 force
        feature ssh
        system default switchport
        system default switchport shutdown
        copp profile strict
      interface mgmt0
      ip address 10.10.99.10 255.255.255.0
      no shutdown
      
      Would you like to edit the configuration? (yes/no) [n]:
      
      Use this configuration and save it? (yes/no) [y]:
      2017 Jun 13 21:24:43 A1 %$ VDC-1 %$ %COPP-2-COPP_POLICY: Control-Plane is protected with policy copp-system-p-policy-strict.
      
      [########################################] 100%
      Copy complete.
      
      User Access Verification
      IP_switch_A_1 login: admin
      Password:
      Cisco Nexus Operating System (NX-OS) Software
      .
      .
      .
      IP_switch_A_1#
  2. Save the configuration:

     IP_switch-A-1# copy running-config startup-config
  3. Reboot the switch and wait for the switch to reload:

     IP_switch-A-1# reload
  4. Repeat the previous steps on the other three switches in the MetroCluster IP configuration.

Downloading and installing the Cisco switch NX-OS software

You must download the switch operating system file and RCF file to each switch in the MetroCluster IP configuration.

About this task

This task requires file transfer software, such as FTP, TFTP, SFTP, or SCP, to copy the files to the switches.

These steps must be repeated on each of the IP switches in the MetroCluster IP configuration.

You must use the supported switch software version.

Steps
  1. Download the supported NX-OS software file.

  2. Copy the switch software to the switch:

    copy sftp://root@server-ip-address/tftpboot/NX-OS-file-name bootflash: vrf management

    In this example, the nxos.7.0.3.I4.6.bin file is copied from SFTP server 10.10.99.99 to the local bootflash:

    IP_switch_A_1# copy sftp://root@10.10.99.99/tftpboot/nxos.7.0.3.I4.6.bin bootflash: vrf management
    root@10.10.99.99's password: password
    sftp> progress
    Progress meter enabled
    sftp> get   /tftpboot/nxos.7.0.3.I4.6.bin  /bootflash/nxos.7.0.3.I4.6.bin
    Fetching /tftpboot/nxos.7.0.3.I4.6.bin to /bootflash/nxos.7.0.3.I4.6.bin
    /tftpboot/nxos.7.0.3.I4.6.bin                 100%  666MB   7.2MB/s   01:32
    sftp> exit
    Copy complete, now saving to disk (please wait)...
  3. Verify on each switch that the switch NX-OS files are present in each switch's bootflash directory:

    dir bootflash:

    The following example shows that the files are present on IP_switch_A_1:

    IP_switch_A_1# dir bootflash:
                      .
                      .
                      .
      698629632    Jun 13 21:37:44 2017  nxos.7.0.3.I4.6.bin
                      .
                      .
                      .
    
    Usage for bootflash://sup-local
     1779363840 bytes used
    13238841344 bytes free
    15018205184 bytes total
    IP_switch_A_1#
  4. Install the switch software:

    install all nxos bootflash:nxos.version-number.bin

    The switch will reload (reboot) automatically after the switch software has been installed.

    The following example shows the software installation on IP_switch_A_1:

    IP_switch_A_1# install all nxos bootflash:nxos.7.0.3.I4.6.bin
    Installer will perform compatibility check first. Please wait.
    Installer is forced disruptive
    
    Verifying image bootflash:/nxos.7.0.3.I4.6.bin for boot variable "nxos".
    [####################] 100% -- SUCCESS
    
    Verifying image type.
    [####################] 100% -- SUCCESS
    
    Preparing "nxos" version info using image bootflash:/nxos.7.0.3.I4.6.bin.
    [####################] 100% -- SUCCESS
    
    Preparing "bios" version info using image bootflash:/nxos.7.0.3.I4.6.bin.
    [####################] 100% -- SUCCESS       [####################] 100%            -- SUCCESS
    
    Performing module support checks.            [####################] 100%            -- SUCCESS
    
    Notifying services about system upgrade.     [####################] 100%            -- SUCCESS
    
    
    
    Compatibility check is done:
    Module  bootable          Impact  Install-type  Reason
    ------  --------  --------------  ------------  ------
         1       yes      disruptive         reset  default upgrade is not hitless
    
    
    
    Images will be upgraded according to following table:
    Module       Image   Running-Version(pri:alt)         New-Version   Upg-Required
    ------  ----------   ------------------------  ------------------   ------------
         1        nxos                7.0(3)I4(1)         7.0(3)I4(6)   yes
         1        bios         v04.24(04/21/2016)  v04.24(04/21/2016)   no
    
    
    Switch will be reloaded for disruptive upgrade.
    Do you want to continue with the installation (y/n)?  [n] y
    
    
    Install is in progress, please wait.
    
    Performing runtime checks.         [####################] 100%    -- SUCCESS
    
    Setting boot variables.
    [####################] 100% -- SUCCESS
    
    Performing configuration copy.
    [####################] 100% -- SUCCESS
    
    Module 1: Refreshing compact flash and upgrading bios/loader/bootrom.
    Warning: please do not remove or power off the module at this time.
    [####################] 100% -- SUCCESS
    
    
    Finishing the upgrade, switch will reboot in 10 seconds.
    IP_switch_A_1#
  5. Wait for the switch to reload and then log in to the switch.

    After the switch has rebooted the login prompt is displayed:

    User Access Verification
    IP_switch_A_1 login: admin
    Password:
    Cisco Nexus Operating System (NX-OS) Software
    TAC support: http://www.cisco.com/tac
    Copyright (C) 2002-2017, Cisco and/or its affiliates.
    All rights reserved.
    .
    .
    .
    MDP database restore in progress.
    IP_switch_A_1#
    
    The switch software is now installed.
  6. Verify that the switch software has been installed:
    show version

    The following example shows the output:

    IP_switch_A_1# show version
    Cisco Nexus Operating System (NX-OS) Software
    TAC support: http://www.cisco.com/tac
    Copyright (C) 2002-2017, Cisco and/or its affiliates.
    All rights reserved.
    .
    .
    .
    
    Software
      BIOS: version 04.24
      NXOS: version 7.0(3)I4(6)   **<<< switch software version**
      BIOS compile time:  04/21/2016
      NXOS image file is: bootflash:///nxos.7.0.3.I4.6.bin
      NXOS compile time:  3/9/2017 22:00:00 [03/10/2017 07:05:18]
    
    
    Hardware
      cisco Nexus 3132QV Chassis
      Intel(R) Core(TM) i3- CPU @ 2.50GHz with 16401416 kB of memory.
      Processor Board ID FOC20123GPS
    
      Device name: A1
      bootflash:   14900224 kB
      usb1:               0 kB (expansion flash)
    
    Kernel uptime is 0 day(s), 0 hour(s), 1 minute(s), 49 second(s)
    
    Last reset at 403451 usecs after  Mon Jun 10 21:43:52 2017
    
      Reason: Reset due to upgrade
      System version: 7.0(3)I4(1)
      Service:
    
    plugin
      Core Plugin, Ethernet Plugin
    IP_switch_A_1#
  7. Repeat these steps on the remaining three IP switches in the MetroCluster IP configuration.

Downloading and installing the Cisco IP RCF files

You must generate and install the RCF file to each switch in the MetroCluster IP configuration.

About this task

This task requires file transfer software, such as FTP, TFTP, SFTP, or SCP, to copy the files to the switches.

These steps must be repeated on each of the IP switches in the MetroCluster IP configuration.

You must use the supported switch software version.

There are four RCF files, one for each of the four switches in the MetroCluster IP configuration. You must use the correct RCF files for the switch model you are using.

Switch

RCF file

IP_switch_A_1

NX3232_v1.80_Switch-A1.txt

IP_switch_A_2

NX3232_v1.80_Switch-A2.txt

IP_switch_B_1

NX3232_v1.80_Switch-B1.txt

IP_switch_B_2

NX3232_v1.80_Switch-B2.txt

Steps
  1. Generate the Cisco RCF files for MetroCluster IP.

    1. Download the RcfFileGenerator for MetroCluster IP

    2. Generate the RCF file for your configuration using the RcfFileGenerator for MetroCluster IP.

      Note Modifications to the RCF files after download are not supported.
  2. Copy the RCF files to the switches:

    1. Copy the RCF files to the first switch:

      copy sftp://root@FTP-server-IP-address/tftpboot/switch-specific-RCF bootflash: vrf management

      In this example, the NX3232_v1.80_Switch-A1.txt RCF file is copied from the SFTP server at 10.10.99.99 to the local bootflash. You must use the IP address of your TFTP/SFTP server and the file name of the RCF file that you need to install.

      IP_switch_A_1# copy sftp://root@10.10.99.99/tftpboot/NX3232_v1.80_Switch-A1.txt bootflash: vrf management
      root@10.10.99.99's password: password
      sftp> progress
      Progress meter enabled
      sftp> get   /tftpboot/NX3232_v1.80_Switch-A1.txt /bootflash/NX3232_v1.80_Switch-A1.txt
      Fetching /tftpboot/NX3232_v1.80_Switch-A1.txt to /bootflash/NX3232_v1.80_Switch-A1.txt
      /tftpboot/NX3232_v1.80_Switch-A1.txt          100% 5141     5.0KB/s   00:00
      sftp> exit
      Copy complete, now saving to disk (please wait)...
      IP_switch_A_1#
    2. Repeat the previous substep for each of the other three switches, being sure to copy the matching RCF file to the corresponding switch.

  3. Verify on each switch that the RCF file is present in each switch's bootflash directory:

    dir bootflash:

    The following example shows that the files are present on IP_switch_A_1:

    IP_switch_A_1# dir bootflash:
                      .
                      .
                      .
    5514    Jun 13 22:09:05 2017  NX3232_v1.80_Switch-A1.txt
                      .
                      .
                      .
    
    Usage for bootflash://sup-local
    1779363840 bytes used
    13238841344 bytes free
    15018205184 bytes total
    IP_switch_A_1#
  4. Configure the TCAM regions on Cisco 3132Q-V and Cisco 3232C switches.

    Note Skip this step if you do not have Cisco 3132Q-V or Cisco 3232C switches.
    1. On Cisco 3132Q-V switch, set the following TCAM regions:

      conf t
      hardware access-list tcam region span 0
      hardware access-list tcam region racl 256
      hardware access-list tcam region e-racl 256
      hardware access-list tcam region qos 256
    2. On Cisco 3232C switch, set the following TCAM regions:

      conf t
      hardware access-list tcam region span 0
      hardware access-list tcam region racl-lite 0
      hardware access-list tcam region racl 256
      hardware access-list tcam region e-racl 256
      hardware access-list tcam region qos 256
    3. After setting the TCAM regions, save the configuration and reload the switch:

      copy running-config startup-config
      reload
  5. Copy the matching RCF file from the local bootflash to the running configuration on each switch:

    copy bootflash:switch-specific-RCF.txt running-config

  6. Copy the RCF files from the running configuration to the startup configuration on each switch:

    copy running-config startup-config

    You should see output similar to the following:

    IP_switch_A_1# copy bootflash:NX3232_v1.80_Switch-A1.txt running-config
    IP_switch-A-1# copy running-config startup-config
  7. Reload the switch:

    reload

    IP_switch_A_1# reload
  8. Repeat the previous steps on the other three switches in the MetroCluster IP configuration.

Setting Forward Error Correction for systems using 25-Gbps connectivity

If your system is configured using 25-Gbps connectivity, you need to set the Forward Error Correction (fec) parameter manually to off after applying the RCF file. The RCF file does not apply this setting.

About this task

The 25-Gbps ports must be cabled prior to performing this procedure.

This task only applies to platforms using 25-Gbps connectivity:

  • AFF A300

  • FAS 8200

  • FAS 500f

  • AFF A250

This task must be performed on all four switches in the MetroCluster IP configuration.

Steps
  1. Set the fec parameter to off on each 25-Gbps port that is connected to a controller module, and then copy the running configuration to the startup configuration:

    1. Enter configuration mode: config t

    2. Specify the 25-Gbps interface to configure: interface interface-ID

    3. Set fec to off: fec off

    4. Repeat the previous steps for each 25-Gbps port on the switch.

    5. Exit configuration mode: exit

      The following example shows the commands for interface Ethernet1/25/1 on switch IP_switch_A_1:

      IP_switch_A_1# conf t
      IP_switch_A_1(config)# interface Ethernet1/25/1
      IP_switch_A_1(config-if)# fec off
      IP_switch_A_1(config-if)# exit
      IP_switch_A_1(config-if)# end
      IP_switch_A_1# copy running-config startup-config
  2. Repeat the previous step on the other three switches in the MetroCluster IP configuration.

Disable unused ISL ports and port channels

NetApp recommends disabling unused ISL ports and port channels to avoid unnecessary health alerts.

  1. Identify the unused ISL ports and port channels:

    show interface brief

  2. Disable the unused ISL ports and port channels.

    You must run the following commands for each identified unused port or port channel.

    SwitchA_1# config t
    Enter configuration commands, one per line. End with CNTL/Z.
    SwitchA_1(config)# int Eth1/14
    SwitchA_1(config-if)# shutdown
    SwitchA_12(config-if)# exit
    SwitchA_1(config-if)# copy running-config startup-config
    [########################################] 100%
    Copy complete, now saving to disk (please wait)...
    Copy complete.