Avi Vantage Integration with VMware Cloud on AWS

Overview

This guide describes the details of integrating Avi Vantage with VMware cloud on AWS. Avi Vantage is deployed as a customer-managed solution in VMC.

Avi Vantage is installed in No Orchestrator mode on VMWare cloud on AWS (VMC).
The deployment of Service Engines on VMC is manual. Once SE is integrated with Avi Controller, virtual service placement and scaling can be handled centrally from the Avi Controller.

The screenshot shown below depicts a typical Avi Vantage deployment with VMC.

The following are the observations from the above diagram:

For Avi Service Engines

Avi Service Engines (SEs) are deployed as virtual machines (VMs) on VMC.
SEs are connected to the logical networks. The following are the two types of logical networks:
- Routed network – over IPsec VPN
- Extended network – over L2 VPN
The SEs connect to the Avi Controller over the management network, which is the logical network connection to the vNIC0 of the SE VM.

For Avi Controller

The Avi Controller cluster is dedicated to the VMC environment or is used for load balancing local vCenters.
Considering the monetary cost of resources on VMC and its ephemeral nature, it is recommended to deploy Avi Controller cluster outside of the VMC environment. However, this is not a restriction.

The following diagram depicts the deployment of Avi Controller cluster and SEs on the VMC infrastructure.

Note: DRS and anti-affinity rules are not applicable for Avi Vantage installed on VMC on AWS.

High Availability

The following are the options currently supported for high availability (in the order of recommendation):

N+M with SE’s in DFW exclusion list (Refer to Manage the Distributed Firewall Exclusion List for more details on distributed firewall exclusion list).
Active/Active with SE’s in DFW exclusion list (Refer to Manage the Distributed Firewall Exclusion List for more details on distributed firewall exclusion list).
Active/standby with MAC masquerade disabled. A two-armed configuration to support Preserve Client IP is supported. For more information, see Deploying NSX Advanced Load Balancer with Inline Topology on VMC on AWS.

Configuration

SE VMs require manual creation. The lack of automation on VMC is because the cloudadmin@vmc.local user does not have all required permissions to read/write to vCenter API and there is no access to the ESX management plane. The access to the ESX management plane is required for Avi automated deployment in on-prem vCenter.

This section covers the following:

Downloading SE image
Uploading SE image to Content Library
Deploying SE VM

Downloading SE Image

Log in to Avi UI as the admin user.
Navigate to Infrastructure > Cloud. Download the SE OVA image using the download icon on the cloud. Use the Default-Cloud or create a new No Orchestrator cloud. Use the following steps to create a new cloud using the No Orchestrator option.

Downloading OVA using the Default-Cloud

Downloading OVA from a No Orchestrator Cloud

Uploading SE Image to Content Library

The downloaded .OVA file is used directly to create an SE VM, but this requires uploading the image to vCenter every time a new VM needs to be created. For faster deployment, the SE image is uploaded to the content library on VMC and can be used multiple times.

Follow the below steps to upload the SE image to the content library.

Login to vCenter and select the option to create a new content library. Provide the name and select the desired vCenter server as shown below.
Select a storage location for the library contents.
Select the Import Item option available on vSphere client and upload the .ovf and .vmdk files as shown below.

Deploying SE VM

Prerequisites

The following data are required for deploying SE VM.

Avi Controller IP address
Authentication token and cloud UUID. Navigate to Infrastructure > Cloud, select the required cloud, and click on the key icon to the generate cloud UUID and the authentication token as shown below.
Management IP address, subnet, and subnet mask. This is required only if DHCP is not enabled on management logical network.

VM Creation

Log in to the vSphere client, select the Templates option, and click on the New VM from this template to create a new VM as shown below.
Select a VM location as shown below.

Optional step: Create a new folder under Workloads to place all the SEs.
Click on the Select a compute resource option to select the resource pool for the deployment.
Click on the Select storage option to select the required datastore.
Click on the Select network option to configure the required networks.

The Management network label (vNIC0) is mapped to the management logical network. The remaining network labels (Data Network 1 – 9) is connected to any of the front-end virtual service’s network or back-end server’s logical network as required. It is left disconnected if not required.
Select the Customize template option to create vApp properties. Provide Avi Controller IP address details, the Cluster UUID, and the authentication token as described in the Prerequisites section.
Review and click on Finish.
Powered on the deployed VM.

Additional Information

To check the newly deployed SE, navigate to Infrastructure > Service Engine tab on the Avi UI.

If the SE VMs are switched on but not connected to the Avi Controller, check firewall ports configured on the Compute Gateway option in VMC console. This option is used to allow management traffic from the SE to the Avi Controller.

For more information on the required ports and protocols, refer to Protocol Ports Used by Avi Vantage for Management Communication.

Note: The SEs open TCP connections to the Avi controller, so the firewall rules should allow outgoing traffic. Since the firewall is stateful, the reverse traffic is automatically allowed.

If the Avi Controller is accessed using public IP address (for example, when it is deployed on another VPC on AWS or if Avi SaaS offering is being used), add NAT rules to allow SE traffic over the internet.

Recommended Configuration

Enabling VMC Deployment in Cloud Configuration

Starting with Avi Vantage version 20.1.5, in the cloud configuration a new knob vmc_deployment is provided to indicate that the cloud is configured in the VMC based deployment. This knob must be set when the deployment is VMC based.

Note: Starting with Avi Vantage version 20.1.5, if vmc_deployment is set, you do not have to configure the TCP Network profile.


[admin:Avi-Controller]: > configure cloud Default-Cloud
[admin:Avi-Controller]: cloud> vmc_deployment
[admin:Avi-Controller]: cloud> save

TCP Network Profile

Prior to Avi Vantage version 20.1.5, you need to configure TCP Network Profile.

As a result of a high number of out-of-order packets within the AWS infrastructure it is recommended to modify the TCP profile that will be applied to virtualservices with a reorder_threshold value set to 8. This modified profile will help with the out-of-order packets potentially improving application performance.

The example below shows creating a brand new tcp profile that will be used by the virtualservices.


[admin:avicontroller]: > configure networkprofile vmc-tcp-proxy
[admin:avicontroller]: networkprofile> profile tcp_proxy_profile
[admin:avicontroller]: networkprofile:profile:tcp_proxy_profile> no automatic
[admin:avicontroller]: networkprofile:profile:tcp_proxy_profile> reorder_threshold 8

[admin:avicontroller]: networkprofile:profile:tcp_proxy_profile> save
[admin:avicontroller]: networkprofile> save
+----------------------------------------+-----------------------------------------------------+
| Field                                  | Value                                               |
+----------------------------------------+-----------------------------------------------------+
| uuid                                   | networkprofile-07ad8f8c-9960-4535-a14c-9065062a58bb |
| name                                   | vmc-tcp-proxy                                       |
| profile                                |                                                     |
|   type                                 | PROTOCOL_TYPE_TCP_PROXY                             |
|   tcp_proxy_profile                    |                                                     |
|     automatic                          | False                                               |
|     ignore_time_wait                   | False                                               |
|     time_wait_delay                    | 2000 milliseconds                                   |
|     max_retransmissions                | 8                                                   |
|     max_syn_retransmissions            | 8                                                   |
|     receive_window                     | 64 kb                                               |
|     nagles_algorithm                   | False                                               |
|     ip_dscp                            | 0                                                   |
|     idle_connection_type               | KEEP_ALIVE                                          |
|     idle_connection_timeout            | 600 sec                                             |
|     use_interface_mtu                  | True                                                |
|     cc_algo                            | CC_ALGO_NEW_RENO                                    |
|     aggressive_congestion_avoidance    | False                                               |
|     reorder_threshold                  | 8                                                   |
|     slow_start_scaling_factor          | 1                                                   |
|     congestion_recovery_scaling_factor | 2                                                   |
|     reassembly_queue_size              | 0                                                   |
|     keepalive_in_halfclose_state       | True                                                |
|     auto_window_growth                 | True                                                |
| connection_mirror                      | False                                               |
| tenant_ref                             | admin                                               |
+----------------------------------------+-----------------------------------------------------+
[admin:avicontroller]: >

Document Revision History

Date	Change Summary
April 15, 2021	Added 'Enabling VMC Deployment in Cloud Configuration' section