Improving Security in your Microservices Architecture with Istio

For our fintech development team, we actively use the microservice approach. Security of inter-service interaction is essential for us.

In this article, I'll explain how Istio works and demonstrate how to use some of its security features with an example. I hope you learn something from this post. Keep on reading!

Why do I require a service network in the first place?

Istio is an open-source service mesh that layers transparently onto existing distributed applications. It combines everything needed to manage and configure inter-service communication:

• Secure service-to-service communication in a cluster with TLS encryption
• Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic
• Fine-grained control of traffic behavior with rich routing rules, retries, failovers, and fault injection
• Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress

And, while there are many open libraries available to implement these functions directly in the service code, Istio allows you to achieve the same results without adding anything to the service itself.

Istio is logically divided into two planes: the data plane and the control plane.

The data plane is a collection of proxy servers (Envoy) that are added to the pod as sidecars. These proxy servers, which are configured from the control plane, provide and control all network communication between microservices.

Service discovery, configuration, and certificate management are all provided by the Management Plane (istiod). It transforms Istio objects into Envoy-compatible configurations and distributes them in the data plane.

You can manually add an envoy to the application pod or set up automatic addition using the Mutating Admission webhook that Istio setup. Put the label istio-injection=enabled on the required namespace to accomplish this.

Istio will add a special init container to the pod, in addition to the proxy sidecar with the envoy, that will redirect combat traffic to the container with envoy. But how is this accomplished? There is no magic in this case, and it is implemented by installing additional iptables rules in the pod's network namespace.

Let's see in practice how the sidecar captures incoming and outgoing traffic from the container. To do this, let's take a look at the network space of a pod with an added Istio sidecar in more detail.

Demo stand:

As an example, I will use a newly created Kubernetes cluster that has Istio installed.

It’s easy to run Kubernetes locally using minikube:

Linux:

curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube
sudo mkdir -p /usr/local/bin/
sudo install minikube /usr/local/bin/
minikube start --driver=<driver_name> // --driver=none will launch all docker containers directly on the local machine.

MacOS:

brew install minikube
minikube start --driver=<driver_name>

You can download Istio’s demo profile from the official website:

curl -L https://istio.io/downloadIstio | sh -
cd istio-1.6.3
export PATH=$PWD/bin:$PATH
istioctl install --set profile=demo

To illustrate inter-service communication, I will use two microservices: product page and details. They both come in the standard Istio delivery to demonstrate their capabilities.

Installing demo microservices:

kubectl label namespace default istio-injection=enabled
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml

Let's take a look at the container list in the product page application:

kubectl -n default get pods productpage-v1-7df7cb7f86-ntwzz -o jsonpath="{.spec['containers','initContainers'][*].name}"
productpage 
istio-proxy 
istio-init

In addition to the product page itself, the pod has a sidecar istio-proxy (the same envoy) and an init container istio-init.

To view the iptables rules configured in the pod namespace, use the nsenter utility. To do this, we need to find out the pid of the container process:

docker inspect k8s_productpage --format '{{ .State.Pid }}'
16286

We can now examine the iptables rules that have been configured in this container.

It is clear that almost all incoming and outgoing traffic is now intercepted and redirected to ports where the envoy is already waiting.

Enabling Mutual Traffic Encryption

Istio enables you to encrypt all traffic between containers, and the applications will be unaware that they are communicating via tls.

Because client certificates are already mounted in the proxy sidecars, Istio does this out of the box with just one manifest.

The algorithm is as follows:

• Before sending requests, the client-side and server-side envoy proxies authenticate each other.
• If the verification is successful, the client proxy encrypts the traffic and sends it to the server proxy.
• The proxy server-side decrypts the traffic and locally redirects it to the actual destination service.

You can enable mTLS at different levels:

• At the network-wide level
• At the namespace level
• At the level of a specific pod

Operating Modes:

• PERMISSIVE: Both encrypted and plain text traffic is allowed;
• STRICT: Only TLS is allowed;

DISABLE: Only plain text is allowed.

Let's access the service details from the product-page pod using curl without TLS enabled and see what comes up in details using tcpdump:

Request:
Dumping the Traffic:

In plain text, the entire body and headings are perfectly readable.

Enable tls. Create an object of the PeerAuthentication type in the namespace with our pods to accomplish this.

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: default
spec:
  mtls:
    mode: STRICT

Let's run the request from the product page to details again and see what we can get:

Authorization:

Istio can configure access from one application to another using authorization policies. Furthermore, unlike traditional Kubernetes network policies, this works at the L7 level. So, For example, you can fine-tune the request methods and paths for HTTP traffic.

As we saw in the previous example, access is granted to all pods in the cluster by default.

Using this yaml file, we will now prohibit all activities in the default namespace:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: deny-all
  namespace: default
spec:
  {}

And let's see if we can get through to the details service:

curl details:9080
RBAC: access denied

Great, now our request fails.

So, now we'll configure access so that only the GET request, and only on the path /details, is allowed, and all other requests are rejected.

There are several possibilities here:

• You can configure requests to pass with specific headers
• By the app's service account
• By outgoing ip address
• By outgoing namespace
• By claims in the JWT token

The easiest thing to maintain is to give access to the application's service account; this has the advantage of not requiring pre-configuration because the demo application bookinfo already comes with the created and mounted service account.

Setting-Up a new access policy:

apiVersion: "security.istio.io/v1beta1"
kind: "AuthorizationPolicy"
metadata:
  name: "details-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: details
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-productpage"]
    to:
    - operation:
        methods: ["GET"]
        paths: ["/details/*"]

And we try to reach out again:

root@productpage-v1-6b64c44d7c-2fpkc:/# curl details:9080/details/0
{"id":0,"author":"William Shakespeare","year":1595,"type":"paperback","pages":200,"publisher":"PublisherA","language":"English","ISBN-10":"1234567890","ISBN-13":"123-1234567890"}

Everything works. Let's try other methods:

root@productpage-v1-6b64c44d7c-2fpkc:/# curl -XPOST details:9080/details/0
RBAC: access denied
root@productpage-v1-6b64c44d7c-2fpkc:/# 
root@productpage-v1-6b64c44d7c-2fpkc:/# curl -XGET details:9080/ping
RBAC: access denied

Conclusion

Finally, I'd like to point out that the features discussed here represent only a small portion of what Istio is capable of. We received and configured inter-service traffic encryption and authorization out of the box but at the cost of adding additional components and, consequently, additional resource consumption.