Skip to main content
NetApp HCI Solutions

KVM on RHEL: NetApp HCI with Cisco ACI

Contributors

KVM (for Kernel-based Virtual Machine) is an open-source full virtualization solution for Linux on x86 hardware such as Intel VT or AMD-V. In other words, KVM lets you turn a Linux machine into a hypervisor that allows the host to run multiple, isolated VMs.

KVM converts any Linux machine into a type-1 (bare-metal) hypervisor. KVM can be implemented on any Linux distribution, but implementing KVM on a supported Linux distribution―like Red Hat Enterprise Linux―expands KVM's capabilities. You can swap resources among guests, share common libraries, and optimize system performance.

Workflow

The following high-level workflow was used to set up the virtual environment. Each of these steps might involve several individual tasks.

  1. Install and configure Nexus 9000 switches in ACI mode, and install and configure APIC software on a UCS C-series server. See the Install and Upgrade documentation for detailed steps.

  2. Configure and set up the ACI fabric by referring to the documentation.

  3. Configure the tenants, application profiles, bridge domains, and EPGs required for NetApp HCI nodes. NetApp recommends using a one-BD-to-one-EPG framework except for iSCSI. See the documentation here for more details. The minimum set of EPGs required are in-band management, iSCSI, VM Motion, VM-data network, and native.

  4. Create the VLAN pool, physical domain, and AEP based on the requirements. Create the switch and interface profiles and policies for vPCs and individual ports. Then attach the physical domain and configure the static paths to the EPGs. See the configuration guide for more details. Also see this table <link> for best practices for integrating ACI with Open vSwitch on the RHEL–KVM hypervisor.

    Note Use a vPC policy group for interfaces connecting to NetApp HCI storage and compute nodes.

  5. Create and assign contracts for tightly-controlled access between workloads. For more details on configuring the contracts, see the guide here.

  6. Install and configure a NetApp HCI Element cluster. Do not use NDE for this installation; rather, install a standalone Element cluster on HCI storage nodes. Then configure the required volumes for the installation of RHEL. Install RHEL, KVM, and Open vSwitch on the NetApp HCI compute nodes. Configure storage pools on the hypervisor using Element volumes for a shared storage service for hosts and VMs. For more details on installation and configuration of KVM on RHEL, see the Red Hat documentation. See the OVS documentation for details on configuring Open vSwitch.

  7. RHEL KVM hypervisor’s Open vSwitch cannot be VMM integrated with Cisco ACI. Physical domain and static paths must be configured on all required EPGs to allow the required VLANs on the interfaces connecting the ACI leaf switches and RHEL hosts. Also configure the corresponding OVS bridges on RHEL hosts and configure VMs to use those bridges. The networking functionality for the RHEL KVM hosts in this solution is achieved using Open vSwitch virtual switch.

Open vSwitch

Open vSwitch is an open-source, enterprise-grade virtual switch platform. It uses virtual network bridges and flow rules to forward packets between hosts. Programming flow rules work differently in OVS than in the standard Linux Bridge. The OVS plugin does not use VLANs to tag traffic. Instead, it programs flow rules on the virtual switches that dictate how traffic should be manipulated before forwarded to the exit interface. Flow rules determine how inbound and outbound traffic should be treated. The following figure depicts the internal networking of Open vSwitch on an RHEL-based KVM host.

Error: Missing Graphic Image

The following table outlines the necessary parameters and best practices for configuring Cisco ACI and Open vSwitch on RHEL based KVM hosts.

Resource Configuration Considerations Best Practices

Endpoint groups

  • Separate EPG for native VLAN

  • Static binding of interfaces towards HCI storage and compute nodes in native VLAN EPG to be on 802.1P mode

  • Static binding of vPCs required on in-band management EPG and iSCSI EPG before KVM installation

  • Separate VLAN pool for physical domain with static allocation turned on

  • Contracts between EPGs to be well defined. Allow only required ports for communication.

  • Use unique native VLAN for discovery during Element cluster formation

Interface Policy

- One vPC policy group per RHEL host
- One vPC policy group per NetApp HCI storage node
- LLDP enabled, CDP disabled

  • NetApp recommends using vPC towards RHV-H hosts

  • Use LACP Active for the port-channel policy

  • Use only Graceful Convergence and Symmetric Hashing control bits for port-channel policy

  • Use Layer4 Src-Port load-balancing hashing method for port-channel policy

  • NetApp recommends using vPC with LACP Active port-channel policy for interfaces towards NetApp HCI storage nodes

Next: ONTAP on AFF: NetApp HCI and Cisco ACI