Skip to main content
ONTAP MetroCluster

Configure MetroCluster IP WAN ISL congestion control

Contributors netapp-aoife
PDFs

Beginning with ONTAP 9.20.1, you can enable WAN Inter-Switch Link (ISL) congestion control in MetroCluster IP configurations so that ISLs running at lower or mixed bandwidths, such as 10Gbps, 25Gbps, or 40Gbps, can be used with platforms that have a MetroCluster IP backend port speed of 100Gbps.

Replication traffic generated by the 100Gbps adapter can overrun ISLs with a lower link speed, causing switch buffer pressure, PAUSE backpressure, packet loss, and potential replication instability. WAN ISL congestion control addresses this by calculating a ceiling for outbound throughput using the ISL bandwidth and node count. The ceiling acts as an outbound throughput threshold that gates interconnect transfer traffic. If measured outbound throughput is above the calculated ceiling, the transfer is rejected, enforcing the outbound throughput threshold on the interconnect network.

Caution Configuring WAN ISL congestion control does not guarantee that performance issues that might be present when using lower bandwidth ISLs will not occur. NetApp strongly recommends that you transition to 100Gbps ISL connectivity for optimal performance.
About this task

  • The MetroCluster IP configuration must be running ONTAP 9.20.1 or later.

  • WAN ISL congestion control is disabled by default and you must explicitly enable it.

  • You need to contact NetApp Support to configure this feature in the following scenarios:

    • If ISL information cannot be obtained automatically when you try to enable or refresh the feature

    • If you have two four-node MetroCluster IP configurations that are connected to the same switches

    • If you need to modify the calculated ceiling after the feature is enabled

  • You must refresh the congestion control configuration to calculate the new ceiling if the topology changes after congestion control is enabled. For example, if the count or speed of the ISLs change, or if you add a new node to the MetroCluster IP configuration.

  • Each site calculates its ceiling independently. When you run the metrocluster interconnect congestion-control enable command on the local site, it triggers the following actions:

    • The local site prepares and calculates its ceiling.

    • The partner site also prepares and calculates the ceiling, separately from the local site where the command is run. While technically unlikely, different ceilings might be calculated for the local and partner sites. This is expected behavior and does not indicate a failure or misconfiguration.

    • Each site writes its calculated value as the desired state, and the kernel then applies it.

  • WAN ISL congestion control operates automatically after it is enabled.

Enable congestion control

To enable WAN ISL congestion control, complete these steps on the local cluster.

Caution If you have two four-node MetroCluster IP configurations that are connected to the same switches, contact NetApp Support to enable this feature.
Steps

  1. Enable WAN ISL congestion control on cluster_A:

    metrocluster interconnect congestion-control enable

    If the command fails with the MCC_CC_ISL_INFO_UNAVAILABLE error, ISL information cannot be obtained automatically and you must contact NetApp Support to enable this feature.

  2. Verify that WAN ISL congestion control is enabled:

    metrocluster interconnect congestion-control show
    cluster_A::> metrocluster interconnect congestion-control show

Node            Enabled Ceiling (MiB/s)
--------------- ------- ---------------
cluster_A-01    true    1250
cluster_A-02    true    1250
2 entries were displayed.
Note If you need to modify the calculated ceiling after this feature is enabled, contact NetApp Support.

Refresh congestion control

To refresh the WAN ISL congestion control configuration, complete these steps on the local cluster. You must refresh the congestion control configuration whenever the topology changes, such as when the count or speed of the ISLs change or when a new node is added to the MetroCluster IP configuration.

Steps

  1. Refresh the WAN ISL congestion control configuration on cluster_A:

    metrocluster interconnect congestion-control refresh

    ONTAP calculates a new ceiling based on the current network conditions and applies it automatically.

    If the command fails with the MCC_CC_ISL_INFO_UNAVAILABLE error, ISL information cannot be obtained automatically and you must contact NetApp Support to refresh your configuration.

Monitor WAN ISL congestion control activity

You can use the following command to verify that congestion control is functioning as expected:

metrocluster interconnect congestion-control show

When enabled, the output should be similar to the following example:

cluster_A::> metrocluster interconnect congestion-control show

Node            Enabled Ceiling (MiB/s)
--------------- ------- ---------------
cluster_A-01    true    1250
cluster_A-02    true    1250
2 entries were displayed.

You can also view more detailed information for each node by using the -instance parameter:

metrocluster interconnect congestion-control show -instance
cluster_A::> metrocluster interconnect congestion-control show -instance

                        Node Name: cluster_A-01
       Congestion Control Enabled: true
       Throughput Ceiling (MiB/s): 1250

                        Node Name: cluster_A-02
       Congestion Control Enabled: true
       Throughput Ceiling (MiB/s): 1250
2 entries were displayed.

Disable congestion control

Steps

  1. Disable WAN ISL congestion control on cluster_A:

    metrocluster interconnect congestion-control disable

  2. Verify that the feature is disabled:

    metrocluster interconnect congestion-control show

    The output should be similar to the following example:

    cluster_A::> metrocluster interconnect congestion-control show

Node            Enabled Ceiling (MiB/s)
--------------- ------- ---------------
cluster_A-01    false   -
cluster_A-02    false   -
2 entries were displayed.