Multiple AKO Instances in a Cluster

Overview

Multiple AKO instances can be deployed in a cluster to create namespace based isolation. This will allow AKO to operate over a group of Kubernetes namespaces, in order to handle objects from these namespaces only. To run multiple AKO, the following features are used:

Namespace Sync Feature

The namespace sync feature allows Kubernetes/OpenShift objects from specific namespaces to be synced with AKO based Avi-Controller. For that, namespace has to be labelled with same key:value pair as that labelKey and labelValue mentioned in values.yaml file.

AKO installation in User Provided Namespace

While installing AKO, using helm, flag -n or --namespace can be used to specify namespace in which AKO has to be installed. If this flag is not specified, AKO will be installed in the avi-system namespace.

Configuring Multiple AKO Instances

In multiple AKO deployment, one AKO instance works as primary AKO instance. This AKO instance is responsible for VRF, static route configuration apart from syncing up Kubernetes/OpenShift objects from set of namespaces.

The flag primaryInstance present in values.yaml denotes whether AKO instance is primary or not. This flag takes boolean true/false value. True indicates AKO instance is primary.

Notes:

  • In multiple AKO deployment, only one AKO instance should be primary.
  • Each AKO should be deployed in a different namespace.

Primary AKO installation

In the following example, primary AKO instance is running in avi-system namespace with namespace sync filter app: migrate. So this AKO will sync up Kubernetes/OpenShift objects from namespaces who has labels app: migrate.


helm install  ako/ako  --generate-name --version 1.10.1 -f /path/to/values.yaml  --set ControllerSettings.controllerHost=<controller IP or Hostname> --set avicredentials.username=<avi-ctrl-username> --set avicredentials.password=<avi-ctrl-password> --set AKOSettings.namespaceSelector.labelKey="app" --set AKOSettings.namespaceSelector.labelValue="migrate" --set AKOSettings.primaryInstance=true --namespace=avi-system

The helm install command without primaryInstance parameter will deploy primary AKO instance.

Non-Primary AKO Installation

In the following example, non-primary AKO instance is running in blue namespace with namespace sync filter key: value2. So this AKO will sync up Kubernetes/OpenShift objects from namespaces who has labels key: value2.


 helm install  ako/ako  --generate-name --version 1.10.1 -f /path/to/values.yaml  --set ControllerSettings.controllerHost=<controller IP or Hostname> --set avicredentials.username=<avi-ctrl-username> --set avicredentials.password=<avi-ctrl-password> --set AKOSettings.namespaceSelector.labelKey="key" --set AKOSettings.namespaceSelector.labelValue="value2" --set AKOSettings.primaryInstance=false --namespace=blue

Few things that to be considered in multiple AKO instances deployment:

  1. All AKO instances should interact with same Avi Controller
  2. Each Kubernetes/OpenShift namespace should be handled by one AKO only
  3. All AKO should be deployed either in ClusterIP or NodePort or NodePortLocal mode

Avi Object Name Convention

  1. For non-primary AKO instance, naming convention for shared VS is: Shared-VS-Name = <cluster-name>–<AKO-namespace>-Shared-L7-<Shard number>. Here <AKO-namespace> is namespace in which AKO pod is deployed.

  2. Non-primary AKO instance will create Avi objects with username = ako-<cluster-name>-<AKO-namespace>