AKS: Open Service Mesh Traffic Access Control

In my previous post AKS: Open Service Mesh & mTLS , I described how to deploy an AKS cluster with Open Service Mesh enabled, and how:

  • Easy is to onboard applications onto the mesh by enabling automatic sidecar injection of Envoy proxy.
  • OSM enables secure service to service communication.

This time I’ll show you that Open Service Mesh (OSM) also provides a nice feature for controlling traffic between microservices: Traffic Access Control based on the SMI specifications .

To control traffic between microservices using OSM you have to be aware of the following custom resource definitions:

  • HTTPRouteGroup: used to describe HTTP/1 and HTTP/2 traffic, enumerating the routes that can be served by an application.
  • TCPRoute: used to describe L4 TCP traffic for a list of ports.
  • UDPRoute: used to describe L4 UDP traffic for a list of ports.
  • TrafficTarget: associates a set of traffic definitions (rules) with a service identity which is allocated to a group of pods. Access is controlled via referenced TrafficSpecs (HTTPRouteGroup, TCPRoute, UDPRoute) and by a list of source service identities.

Access is controlled based on service identity, at present the method of assigning service identity is using Kubernetes service accounts, provision for other identity mechanisms will be handled by the spec at a later date.

A valid TrafficTarget must specify a destination, at least one rule, and at least one source.

Now let’s see how this works:

Deploy an AKS cluster with OSM enabled

Follow the steps in my previous post AKS: Open Service Mesh & mTLS to deploy an AKS cluster with Open Service Mesh enabled.

Disable Permissive Traffic Policy Mode

By default the OSM service mesh is configured to allow all traffic between services, so let’s disable the Permissive Traffic Policy Mode by running:

1kubectl patch meshconfig osm-mesh-config -n kube-system -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":false}}}' --type=merge

Double check that the Permissive Traffic Policy Mode is disabled:

1kubectl get meshconfig osm-mesh-config -n kube-system -o yaml | grep -i enablePermissiveTrafficPolicyMode

Deploy the sample microservices

Create nginx.yaml with the following contents:

 1---
 2apiVersion: v1
 3kind: ServiceAccount
 4metadata:
 5  creationTimestamp: null
 6  name: nginx
 7  namespace: default
 8
 9---
10apiVersion: v1
11kind: Pod
12metadata:
13  name: nginx
14  labels:
15    run: nginx
16  annotations:
17    openservicemesh.io/sidecar-injection: enabled
18spec:
19  containers:
20  - image: nginx
21    name: nginx
22    ports:
23    - containerPort: 80
24  serviceAccountName: nginx
25
26---
27apiVersion: v1
28kind: Service
29metadata:
30  creationTimestamp: null
31  name: nginx
32spec:
33  ports:
34  - port: 80
35    protocol: TCP
36    targetPort: 80
37  selector:
38    run: nginx

Note that:

  • The openservicemesh.io/sidecar-injection: enabled annotation is required to enable sidecar injection.
  • The service account nginx is used to run the nginx container and will be key when defining the traffic rules.

Create busybox.yaml with the following contents:

 1---
 2apiVersion: v1
 3kind: ServiceAccount
 4metadata:
 5  creationTimestamp: null
 6  name: busybox
 7  namespace: default
 8
 9---
10apiVersion: v1
11kind: Pod
12metadata:
13  name: busybox
14  labels:
15    run: busybox
16  annotations:
17    openservicemesh.io/sidecar-injection: enabled
18spec:
19  containers:
20  - image: busybox
21    name: busybox
22    ports:
23    - containerPort: 80
24    command:
25    - sh
26    - -c
27    - sleep 3600
28  serviceAccountName: busybox

Note that:

  • The openservicemesh.io/sidecar-injection: enabled annotation is required to enable sidecar injection.
  • The service account busybox is used to run the busybox container and will be key when defining the traffic rules.

Deploy the microservices:

1kubectl apply -f nginx.yaml
2kubectl apply -f busybox.yaml

Test Connectivity

Since we disabled the Permissive Traffic Policy Mode, traffic is not allowed between the microservices. Test the connectivity by running the following:

1kubectl exec -it busybox -c busybox -- sh

once inside the container, run:

1wget -O- http://nginx

The result should be similar to the following:

1Connecting to nginx (10.0.149.72:80)
2wget: error getting response: Resource temporarily unavailable

Add Traffic Access Control to allow traffic between microservices

Let’s fix the issue by adding Traffic Access Control to the mesh and allow communication between the microservices:

Create nginx_traffic_target.yaml with the following contents:

 1apiVersion: specs.smi-spec.io/v1alpha4
 2kind: HTTPRouteGroup
 3metadata:
 4  name: all-routes
 5  namespace: default
 6spec:
 7  matches:
 8  - name: everything
 9    pathRegex: "/*"
10    methods: ["*"]
11
12---
13apiVersion: access.smi-spec.io/v1alpha3
14kind: TrafficTarget
15metadata:
16  name: nginx
17  namespace: default
18spec:
19  destination:
20    kind: ServiceAccount
21    name: nginx
22    namespace: default
23  rules:
24  - kind: HTTPRouteGroup
25    name: all-routes
26    matches:
27    - everything
28  sources:
29  - kind: ServiceAccount
30    name: busybox
31    namespace: default

Note that:

  • The HTTPRouteGroup all-routes allows all traffic and methods, but you could easily make it more restrictive.
  • The TrafficTarget associates the destination and source with the rules.
  • The TrafficTarget defines sources and destination by using service account names: nginx and busybox.

Deploy the HTTPRouteGroup and the TrafficTarget

Run:

1kubectl apply -f nginx_traffic_target.yaml

Recheck Connectivity

Now that we explicitly allowed traffic between the microservices, we can test connectivity by running the following:

1kubectl exec -it busybox -c busybox -- sh

and once inside the container, run the following command to test connectivity:

1wget -O- http://nginx

Hope it helps!!!

Please find the complete sample here

References:


AKS: Open Service Mesh & mTLS
AKS: Resize Private Volume Claim to expand a Managed Premium Disk
comments powered by Disqus