Envoy

envoy

Load balancing is a common term for a devops engineer. When huge of traffic comes into to your system, you need to find out a way to scale the system so that it can handle it properly. One solution is to increase the performance of the running single node. Another solution is to add more nodes and distribute the work among these nodes. Having many node have another added advantage of high availability. Load balancers used to distribute the traffic among the worker nodes. Envoy proxy is a proxy service that used in latest trending concept that known as Service Mesh. We will see the load balancing aspect of the Envoy Proxy in this blog post. Load Balancers Load balancers is an endpoint that listen to the request that coming into the computation cluster. When request comes into the Load balancer, it check for available worker nodes and distribute request among worker nodes. Load balancers do following things. Service Discovery: Check available worker nodes Health check: Regularly inspect worker nodes health. Load balancing: Distribute the request between the worker nodes. Proxy Proxy is an intermediary component that exist between two endpoints. Proxy service take client request and forward it to the destination server. There are two types of proxies. Forward proxy and Reverse Proxy. Instead of sending request directly to the endpoint, we can also send it through a proxy. This type of proxy known as Forward proxy. Forward proxy commonly used to bypass the firewall restrictions and access to blocked websites. Revers proxy is a type of a proxy service that take incoming client request and forward it to the server that can fulfill it. The result will be rout back to the client. Addition to that, proxy also provide more control over the client request. It can also cache the request and speedup the network performance. Reverse Proxy used to To enable indirect access when a website disallows direct connections as a security measure. To stream internal content to Internet users. To allow for load balancing between severs. To disable access to a site. Load balancing topologies Proxy sitting between client and backend endpoint. Load balancing can be divided into following topologies based on the place where proxy service placed. Middle Proxy All client request comes into the the middle proxy. Middle proxy rout request into the worker node. This type of load balancers are simple and straight forward. Embedded Client Library The biggest problem in the Middle Proxy is that, Single point failure. If Middle Proxy server get down, then client services unable to access backend services.In this type of proxy, instead of central load balancer, load balancing done by the client it self. This kind of system can implemented by using gRPC libraries. Growing complexity become a problem in this type of load balancers. Also, developer need to install load balancing component for each of the service. Side Car Proxy The biggest problem in Embedded Client Library is that the complexity of building communication component for each of the services. With the recent trend of using container technology, Client Library separated into the containers. So, there is no programming languages lock in while developing decentralized load balancers. This is know as Side Car. This type of proxy service implementation known as Service Mesh. Side Car responsible to rout client request into the appropriate backend service. Envoy is high performance revers proxy written in C++ language by Lyft. Envoy used to interconnect services in Service Mesh. Here follow are common terminology that used by Envoy proxy. Host: An entity capable of network communication. Downstream: Hosts that send request to the envoy proxy. Upstream: Host that receive request from the envoy proxy. Listener: Named network location that can connect to an envoy proxy through a downstream. Cluster: Cluster is group of logically same upstream host that envoy can connect. Envoy can discover cluster by using service discovery. Front Envoy Proxy Aport from Side Car proxy, Envoy can be configure as Front envoy proxy as well. Front proxy configured as primary load balancer to the request coming in from the public internet. This proxy also know and edge proxy. Overall architecture of Service Mesh would be like follows. Here, front proxy used as load balancer for the incoming internet traffic. Here TLS termination also perform. Then request rout to the relevant services through side car proxies. Service mesh can detect available services through service discovery. It also provide circuit brake features to handle fail overs. Collectively, Envoy provide whole bunch of features to implement a Service Mesh. Types of Load Balancers in Envoy Proxy When proxy need to acquire connection to host in upstream cluster, the cluster manager use following polices to rout traffic. Round Robbin Rout load to each of the worker nodes(upstream host) circular order. All worker node consider as same and all node get same amount of load. Random Select worker node by random and rout the traffic. This is know to be perform better than Round Robbin policy. Weighted Least Request This policy based on the number of connection that are keep while loading balance. Assume there are two worker nodes with same specs. Due to some reason first worker node take longer time to response. So it also have to keep it connection to first worker node longer than second node. In this scenario, load balancer can put more weight on second worker node rather sending traffic into the first node. Original Destination This type of load balancer used when a given connection needs to connect to some particular upstream host. host selected by reading client’s meta data. Other than load balancing, Envoy also provide following feature to implement Service Mesh. Dynamic service discovery TLS termination HTTP/2 and gRPC proxies Circuit breakers Health checks Staged rollouts with %-based traffic split Fault injection Rich metrics We will go through each of these features in next article. This article is to give you the basic introduction about Envoy Proxy and how it do Load Balancing. See you in another article. Cheers :) References https://devform.netlify.com/introduction-to-envoy-proxy-load-balancing/ https://www.envoyproxy.io/docs/envoy/v1.5.0/intro/arch_overview/load_balancing https://www.jscape.com/blog/load-balancing-algorithms https://blog.envoyproxy.io/introduction-to-modern-network-load-balancing-and-proxying-a57f6ff80236

Gentle Introduction to the Envoy Proxy and Load-balancing

The Noonification: Laravel Envoy - How to Automate the Boring Stuff (7/10/2024)

Laravel Envoy - How to Automate the Boring Stuff

লারাভেল দূত - বিরক্তিকর জিনিসগুলি কীভাবে স্বয়ংক্রিয় করা যায়

Laravel Envoy – So automatisieren Sie den langweiligen Kram

Laravel Envoy: cómo automatizar las cosas aburridas

Laravel Envoy - Comment automatiser les choses ennuyeuses

लारवेल एन्वॉय - बोरिंग चीजों को स्वचालित कैसे करें

Laravel Envoy - 退屈な作業を自動化する方法

Laravel Envoy - 지루한 작업을 자동화하는 방법

Laravel Envoy - Como automatizar coisas chatas

Laravel Envoy — Как автоматизировать скучные вещи

Laravel Elçisi - Sıkıcı Şeyler Nasıl Otomatikleştirilir

Laravel Envoy - Cách tự động hóa những công việc nhàm chán

Laravel Envoy - 如何自动化无聊的事情

The Salvaging of Civilization, by H. G. Wells, is part of the HackerNoon Books Series. You can jump to any chapter in this book here . THE ENVOY THE ENVOY In the preceding papers I have, with some repetition and much stumbling, set out a fairly complete theory of what men and women have to do at the present time if human life is to go on hopefully to any great happiness and achievement in the days to come. Much of this material was first prepared to be delivered to a lecture audience, and I regret that ill-health has prevented a complete re-writing of these portions. There is more of the uplifted forefinger and the reiterated point than I should have allowed myself in an essay. But this is a loss of grace rather than of clearness. And since I am stating a case and not offering the reader anything professing to be a literary work, I shall not apologise for finally summing up and underlining the chief points of this book. They are, firstly: that a great change in human conditions has been brought about during the past century, and secondly that a vast task of adaptation, which must be, initially and fundamentally, mental adaptation, has to be undertaken by our race. It is a task which politicians, who live from day to day, and statesmen, who live from event to event, may hinder or aid very greatly, but which they cannot be expected to conduct or control. Politicians and statesmen perforce live and work in the scheme of ideas they find about them; the conditions of their activities are made for them. They can be compelled by the weight of public opinion to help it, but the driving force for this great task must come not from official sources but from the steadfast educational pressure of a great and growing multitude of convinced people. In times of fluctuation and dissolving landmarks, the importance of the teacher—using the word in its widest sense—rises with the progressive dissolution of the established order. [Pg 194] The creative responsibility for the world to-day passes steadily into the hands of writers and school teachers, students of social and economic science, professors and poets, editors and journalists, publishers and newspaper proprietors, preachers, every sort of propagandist and every sort of disinterested person who can give time and energy to the reconstruction of the social idea. Human life will continue to be more and more dangerously chaotic until a world social idea crystallizes out. That—and no existing institution and no current issue—is the primary concern of the present age. We need, therefore, before all other sorts of organization, educational organizations; we need, before any other sort of work, work of education and enlightenment; we need everywhere active societies pressing for a better, more efficient con duct of public schooling, for a wider, more enlightening school curriculum, for a world-wide linking-up of educational systems, for a ruthless subordination of naval, military and court expenditure to educational needs, and for a systematic discouragement of mischief-making between nation and nation and race and race and class and class. I could wish to see Educational Societies, organized as such, springing up everywhere, watching local bodies in order to divert economies from the educational starvation of a district to other less harmful saving; watching for obscurantism and reaction and mischievous nationalist teaching in the local schools and colleges and in the local press; watching members of parliament and congressmen for evidences of educational good-will or malignity; watching and getting control of the administration of public libraries; assisting, when necessary, in the supply of sound literature in their districts; raising funds for invigorating educational propaganda in poor countries like China and in atrociously educated countries like Ireland, and corresponding with kindred societies throughout the world. I believe such societies would speedily become much more influential than the ordinary political party clubs and associations that now use up so much human energy in the western communities. Subordinating all vulgar political considerations to educational development as the supreme need in the world's affairs, even quite small societies could exercise a powerful decisive voice in a great number of political contests. And an educational movement is more tenacious than any other sort of social or political movement whatever. It trains its adherents. What it wins it holds. [Pg 195] [Pg 196] I know that in thus putting all the importance upon educational needs at the present time I shall seem to many readers to be ignoring quite excessively the profound racial, social and economic conflicts that are in progress. I do. I believe we shall never get on with human affairs until we do ignore them. I offer no suggestion whatever as to what sides people should take in such an issue as that between France and Germany or between Sinn Fein and the British Government, or in the class war. I offer no such suggestion because I believe that all these conflicts and all such current conflicts are so irrational and destructive that it is impossible for a sane man who wishes to serve the world to identify himself with either side in any of them. These conflicts are mere aspects of the gross and passionate stupidity and ignorance and sectionalism of our present world. The class war, the push for and the resistance to some vague reorganization called the Social Revolution—such things are the natural inevitable result of the sordid moral and intellectual muddle of our common ideas about property. The capitalist, the employer, the property-owning class, as a class, have neither the intelligence nor the conscience to comprehend any moral limitations, any limitations whatever but the strong arm of the law, upon what they do with their property. Their black and obstinate ignorance, the clumsy adventurousness they call private enterprise, their unconscious insolence to poor people, their stupidly conspicuous self-indulgence, produce as a necessary result the black hatred of the employed and the expropriated. On one side we have greed, insensibility and incapacity, on the other envy and suffering stung to vindictive revolt: on neither side light nor generosity nor creative will. Neither side has any power to give us any reality we need. Neither side is more than a hate and an aggression. How can one take sides between them? [Pg 197] The present system, unless it can develop a better intelligence and a better heart, is manifestly destined to foster fresh wars and to continue wasting what is left of the substance of mankind, until absolute social disaster overtakes us all. And manifestly the revolutionary communist, at his present level of education, has neither the plans nor the capacity to substitute any more efficient system for this crazy edifice of ill-disciplined private enterprise that is now blundering to destruction. But at a higher level of intelligence, at a level at which it is possible to define the limitations of private property clearly and to ensure a really loyal and effectual co-operation between individual and state, this issue—this wholly destructive conflict between the property manipulator and the communist fanatic which is now rapidly wrecking our world—disappears. It disappears as completely as the causes of a murderous conflict between two drunken men will disappear when they are separated and put under a stream of clear cold water. [Pg 198] So it is that, in spite of their apparent urgency, I ask the reader to detach himself from these present conflicts of national politics, of political parties and of the class war as completely as he can; or, if he cannot detach himself completely, then to play such a part in them, regardless of any other consideration, as may be most conducive to a wide-thinking, wide-ranging education upon which we can base a new world order. A resolute push for quite a short period now might reconstruct the entire basis of our collective human life. In this book I have tried to show what form that push should take, to show that it has a reasonable hope of an ultimate success, and that unless it is made, the outlook for mankind is likely to become an entirely dismal prospect. I put these theses before the reader for his consideration. They are not discursive criticisms of life, not haphazard grumblings at our present discontents, they are offered as the fundamental propositions of an ordered constructive project in which he can easily find a part to play commensurate with his ability and opportunities. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. H. G. Wells (2010). The Salvaging of Civilization. Urbana, Illinois: Project Gutenberg. Retrieved October 2022, from https://www.gutenberg.org/files/33889/33889-h/33889-h.htm This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org , located at https://www.gutenberg.org/policy/license.html.

The Envoy 

The Wheels of Chance by H. G. Wells, is part of the HackerNoon Books Series. You can jump to any chapter in this book here . THE ENVOY XLI. THE ENVOY So the story ends, dear Reader. Mr. Hoopdriver, sprawling down there among the bracken, must sprawl without our prying, I think, or listening to what chances to his breathing. And of what came of it all, of the six years and afterwards, this is no place to tell. In truth, there is no telling it, for the years have still to run. But if you see how a mere counter-jumper, a cad on castors, and a fool to boot, may come to feel the little insufficiencies of life, and if he has to any extent won your sympathies, my end is attained. (If it is not attained, may Heaven forgive us both!) Nor will we follow this adventurous young lady of ours back to her home at Surbiton, to her new struggle against Widgery and Mrs. Milton combined. For, as she will presently hear, that devoted man has got his reward. For her, also, your sympathies are invited. The rest of this great holiday, too—five days there are left of it—is beyond the limits of our design. You see fitfully a slender figure in a dusty brown suit and heather mixture stockings, and brown shoes not intended to be cycled in, flitting Londonward through Hampshire and Berkshire and Surrey, going economically—for excellent reasons. Day by day he goes on, riding fitfully and for the most part through bye-roads, but getting a few miles to the north-eastward every day. He is a narrow-chested person, with a nose hot and tanned at the bridge with unwonted exposure, and brown, red-knuckled fists. A musing expression sits upon the face of this rider, you observe. Sometimes he whistles noiselessly to himself, sometimes he speaks aloud, “a juiced good try, anyhow!” you hear; and sometimes, and that too often for my liking, he looks irritable and hopeless. “I know,” he says, “I know. It's over and done. It isn't IN me. You ain't man enough, Hoopdriver. Look at yer silly hands!... Oh, my God!” and a gust of passion comes upon him and he rides furiously for a space. Sometimes again his face softens. “Anyhow, if I'm not to see her—she's going to lend me books,” he thinks, and gets such comfort as he can. Then again; “Books! What's books?” Once or twice triumphant memories of the earlier incidents nerve his face for a while. “I put the ky-bosh on HIS little game,” he remarks. “I DID that,” and one might even call him happy in these phases. And, by-the-bye, the machine, you notice, has been enamel-painted grey and carries a sonorous gong. This figure passes through Basingstoke and Bagshot, Staines, Hampton, and Richmond. At last, in Putney High Street, glowing with the warmth of an August sunset and with all the 'prentice boys busy shutting up shop, and the work girls going home, and the shop folks peeping abroad, and the white 'buses full of late clerks and city folk rumbling home to their dinners, we part from him. He is back. To-morrow, the early rising, the dusting, and drudgery, begin again—but with a difference, with wonderful memories and still more wonderful desires and ambitions replacing those discrepant dreams. He turns out of the High Street at the corner, dismounts with a sigh, and pushes his machine through the gates of the Antrobus stable yard, as the apprentice with the high collar holds them open. There are words of greeting. “South Coast,” you hear; and “splendid weather—splendid.” He sighs. “Yes—swapped him off for a couple of sovs. It's a juiced good machine.” The gate closes upon him with a slam, and he vanishes from our ken. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. H. G. Wells (2009). The Wheels of Chance. Urbana, Illinois: Project Gutenberg. Retrieved October 2022, from https://www.gutenberg.org/files/1264/1264-h/1264-h.htm This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org, located at https://www.gutenberg.org/policy/license.html.

THE ENVOY

In , I explained what CRD is and how it can be useful to solve the problem. In this article, I will show you how you can write a controller that will monitor changes in custom resources. And in the next article, the controller will already begin to respond to these changes and will configure the Envoy. the last article So, what is a controller? The controller is a program that communicates with Kubernetes through its API. When in the last article, I announced a new type of Custom Resources and announced a new API for it: . proxy.company.com/v1 To interact with the Kubernetes API already, have a lot of ready-made clients in different programming languages, but I will create the examples of golang. Especially since, for golang, there is even an "official" . That’s what I plan to use. code generator Entities The first thing I need to do is create entities to which the Sidecar resource data from the API will be mapped. For this, we need two packages: Provides all necessary abstracts and interfaces k8s.io/apimachinery API client for golang k8s.io/client-go Let me remind you that in the last article, I announced a new type of Sidecar resource. Now, I will describe the structure for it in golang using the packages listed above in the file . types.go package v1

import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

type SidecarSpec struct {
	AppName string `json:"appName"`
	Cb      struct {
		Timeout int `json:"timeout"`
		Tries   int `json:"tries"`
	}
}

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

type Sidecar struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec SidecarSpec `json:"spec"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

type SidecarList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata"`

	Items []Sidecar `json:"items"`
} SidecarSpec matches the description from , recall what it looks like. Custom Resource Definition spec:
  type: object
    properties:
      appName:
        type: string
      cb:
        type: object
        properties:
          timeout:
            type: integer 
          tries:
            type: integer contains additional fields in which various meta-information must be collected. is a structure for a set of structures. For cases when a list of sidecars is requested from the API, his fields for meta-information are slightly different. Sidecar SidecarList Sidecar I will also need the file to register the entity. register.go package v1

import (
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/schema"
)

// SchemeGroupVersion is group version used to register these objects
var SchemeGroupVersion = schema.GroupVersion{
	Group:   "proxy.company.com",
	Version: "v1",
}

// Kind takes an unqualified kind and returns back a Group qualified GroupKind
func Kind(kind string) schema.GroupKind {
	return SchemeGroupVersion.WithKind(kind).GroupKind()
}

// Resource takes an unqualified resource and returns a Group qualified GroupResource
func Resource(resource string) schema.GroupResource {
	return SchemeGroupVersion.WithResource(resource).GroupResource()
}

var (
	// SchemeBuilder initializes a scheme builder
	SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)
	// AddToScheme is a global function that registers this API group & version to a scheme
	AddToScheme = SchemeBuilder.AddToScheme
)

// Adds the list of known types to Scheme.
func addKnownTypes(scheme *runtime.Scheme) error {
	scheme.AddKnownTypes(SchemeGroupVersion,
		&Sidecar{},
		&SidecarList{},
	)
	metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
	return nil
} I specified the API name and version. Group:   "proxy.company.com",
Version: "v1", And added two and entities to the known data schemes. It is important to place the file according to the name and version of . Sidecar SidecarList internal/apis/proxy.company.com/v1 Generator The main task of the generator - on the basis of the described entities to generate a code, which in essence can make queries to the Kubernetes API and a mapped response to the described data structures. Generator downloaded as standard. go get k8s.io/code-generator But since the generator will not import into any of the packages, it will not actually be downloaded to the vendor directory. (I prefer to use this directory to work for more convenient debugging). So, I used a to create a file with a generator import. little hack package hack

import _ "k8s.io/code-generator" And now, when doing , the generator is in vendor and I have the ability to use its bash script generate-groups.sh, which has already solved most of the possible needs of developers. I’ll give you the right away. go mod download launch code SCRIPT_ROOT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )/.." &> /dev/null && pwd )

bash vendor/k8s.io/code-generator/generate-groups.sh \
	"deepcopy,client,informer" \
	github.com/antgubarev/xds-controller/internal/generated \
	github.com/antgubarev/xds-controller/internal/apis \
	proxy.company.com:v1 \
	--output-base=$SCRIPT_ROOT/../../.. \
	--go-header-file=$SCRIPT_ROOT/hack/boilerplate.go.txt Argument assignment The entities I need to generate. You can also simply specify all, and then all possible will be generated. The path by which the generated files will be placed. The path where the data scheme from which the generation takes place. API name and version. It is important to coincide with the way where the structures are placed, as I wrote above. There are also two options in the generator. The root directory into which the generated files will be added. That is to say, the + In my case, this value will be --output-base --output-base github.com/antgubarev/xds-controller/internal/generated GOPATH/src The header file if you want to place any additional information in each generated file. --go-header-file So, let’s run it and look at the result: ./hack/upd.sh                                                               15ms (master*)
Generating deepcopy funcs
Generating clientset for proxy.company.com:v1 at github.com/antgubarev/xds-controller/internal/generated/clientset
Generating listers for proxy.company.com:v1 at github.com/antgubarev/xds-controller/internal/generated/listers
Generating informers for proxy.company.com:v1 at github.com/antgubarev/xds-controller/internal/generated/informers 3 subdirectories appeared in the directory internal/generated Client for API queries clientset A set of components for tracking changes informers Informer is needed and in this article, it is of no additional interest listers This set will be sufficient for the current task so far. Controller Now, I can start tracking changes in Sidecar resources. You can see the full controller code . I’ll describe it in more detail. here import versionedclientset "github.com/antgubarev/xds-controller/internal/generated/clientset/versioned"

//...

cfg, err := clientcmd.BuildConfigFromFlags("", filepath.Join(homedir.HomeDir(), ".kube", "config"))
if err != nil {
	logger.Fatalf("BuildConfigFromFlags: %v", err)
}

_, err = kubernetes.NewForConfig(cfg)
if err != nil {
	logger.Fatalf("Error building kubernetes clientset: %s", err.Error())
}

proxyVersionedClient, err := versionedclientset.NewForConfig(cfg)
if err != nil {
	logger.Fatalf("Error building example clientset: %s", err.Error())
} First, I need to create a client. I used the config for , which is in the home directory, usually, it’s . Next, I used the generated code from . I can already create, delete, and edit Sidecar custom resources using this client. kubectl ~/. kube/config internal/generated/clientset The entire generated code works with the data structures from the ; it has done a lot of work for us. internal/apis generator proxyInformerFactory := proxyinformers.NewSharedInformerFactory(proxyVersionedClient, time.Second*60)

idecarsInformer := proxyInformerFactory.Proxy().V1().Sidecars().Informer()
sidecarsInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
	AddFunc: func(obj interface{}) {
		sidecar, ok := obj.(*v1.Sidecar)
		if !ok {
			logger.Errorf("Can't parse added object: %v", obj)
		}
		logger.Infof("Sidecar %s is added", sidecar.Name)
	},
	UpdateFunc: func(oldObj interface{}, newObj interface{}) {
		switch newObj.(type) {
		case *v1.Sidecar:
			newSidecar, ok := newObj.(*v1.Sidecar)
			if !ok {
				logger.Errorf("Can't parse updated object: %v", newSidecar)
			}
			if oldObj != newObj {
				logger.Infof("Sidecar %s is updated", newSidecar.Name)
			}
		default:
			logger.Errorf("update event: unknown object %v", newObj)
		}
	},
	DeleteFunc: func(obj interface{}) {
		sidecar, ok := obj.(*v1.Sidecar)
		if !ok {
			logger.Errorf("Can't parse deleted object: %v", obj)
		}
		logger.Infof("Sidecar %s is deleted", sidecar.Name)
	},
}) Informer is also a client, but it is designed to inform the client about new updates. In the code above, I announced the handler for 3 events created a new Sidecar resource AddFunc updated Sidecar range resource UpdateFunc removed Sidecar resource DeleteFunc So far, these features do not do anything useful, only log events in the console and later we will definitely see how it works. stop := make(chan struct{})
defer close(stop)

proxyInformerFactory.Start(stop)
if !cache.WaitForCacheSync(stop, sidecarsInformer.HasSynced) {
	logger.Info("Failed to sync cache")
}

<-stop I called the function which creates a go routine to track events. The client stores a cache with the required objects so as not to query them every time in real-time. He also gave me a channel to get the program done properly. Start To be sure of the integrity and reliability of this cache in the previous example, the interval of the full update was set to . Note that this procedure generates refresh events for those data that are already in the cache (because they will be updated in the cache). time.Second*60 In order to avoid unnecessary events, I perform a check on the functions UpdateFunc if oldObj != newObj {
	logger.Infof("Sidecar %s is updated", newSidecar.Name)
} Now, we can test how it works. First, I run the program and wait for readiness (all connections I executed with locally running minikube). I created a with 3 new resources. manifest apiVersion: proxy.company.com/v1
kind: Sidecar
metadata:
  name: service1
spec:
  appName: service1
  cb:
    timeout: 5000
    tries: 3

---
apiVersion: proxy.company.com/v1
kind: Sidecar
metadata:
  name: service2
spec:
  appName: service2
  cb:
    timeout: 500
    tries: 1

---
apiVersion: proxy.company.com/v1
kind: Sidecar
metadata:
  name: service3
spec:
  appName: service3
  cb:
    timeout: 3000
    tries: 3 Then I applied the manifest (controller is running). kubectl apply -f services.cr.yaml                                        
sidecar.proxy.company.com/service1 created
sidecar.proxy.company.com/service2 created
sidecar.proxy.company.com/service3 created Output INFO[0015] Sidecar service1 is added
INFO[0015] Sidecar service2 is added
INFO[0015] Sidecar service3 is added That’s right. Three resources were created and triggered the add event for each resource created. Now, I have changed the parameters of the variables in the first two resources to any other values and applied the manifest again. kubectl apply -f services.cr.yaml                                           
sidecar.proxy.company.com/service1 configured
sidecar.proxy.company.com/service2 configured
sidecar.proxy.company.com/service3 unchanged Output INFO[0011] Sidecar service1 is updated
INFO[0011] Sidecar service2 is updated Everything is working properly again. And now, I will try to remove one of the resources. kubectl delete sidecar service1                                             
sidecar.proxy.company.com "service1" deleted Output INFO[0014] Sidecar service1 is deleted Conclusion I showed a way to track the changes that occur with custom resources in Kubernetes. In the next article, I will use these events to configure a sidecar based on Envoy via the xDS protocol. You can see the full example in the . repository If you like my articles, please encourage me in the . Noonies Photo by on C Dustin Unsplash

Using the Kubernetes Controller for Envoy (Part 2)

In this article, I open a series that will consider the approach to the Envoy configuration with Kubernetes CRD. I will tell about the implementation of the controller for Kubernetes, which watch custom resources in the Kubernetes cluster and performs the Envoy configuration based on this data. This approach is used for example in . After reading the article you will also be able to better understand how it works. Istio The xDS protocol allows Envoy to dynamically configure proxy rules without rebooting the proxy itself and changing its settings. This gives many advantages in modern multi-component and distributed systems. Services can be created and deleted, become temporarily unavailable, or authentication rules can be changed and much more. All this must be done instantly in the production environment. It is important not to change the application code because the infrastructure can change even faster than the application itself. Custom Resource Definition So the first thing we’re going to talk about is the Custom Resource Definition. It’s kind of like a table in a database. You first describe the schematic of this table, and then you manipulate the elements: create, update, delete. Within the Kubernetes cluster, we can also track events with these elements and react to those events. In my example, I will manage an object that should contain behavior information on the circuit breaker pattern Connection timeout Number of tries before the error is returned That is, if you receive a request, the envoy will make three attempts to get a response from the backend before returning the error code. This can be useful when we know in advance that the backend does not respond to an average of every tenth request. Custom Resource Definition apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: sidecars.proxy.company.com
spec:
  group: proxy.company.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                appName:
                  type: string
                cb:
                  type: object
                  properties:
                    timeout:
                      type: integer 
                    tries:
                      type: integer
  scope: Cluster
  names:
    plural: sidecars 
    singular: sidecar 
    kind: Sidecar
    shortNames:
      - sc . Group for the resource type. Must contain the domain. group: proxy.company.com . Name of a new resource type. Formed by name: sidecars.proxy.company.com {resource_name}. {domain} . Here I specify all possible names of the resource type. In singular, plural, etc. These names will be used when working with . names kubectl Note that the schematic description takes place inside the versions block. This is important as the schematic is likely to change. So let’s apply the new configuration. kubectl apply -f crd.yaml
customresourcedefinition.apiextensions.k8s.io/sidecars.proxy.company.com created And now I can manage objects (Custom resource or CR) with a new type of proxy resource that I defined. apiVersion: proxy.company.com/v1
kind: Sidecar
metadata:
  name: service1
spec:
  appName: service1
  cb:
    timeout: 5000
    tries: 3

---
apiVersion: proxy.company.com/v1
kind: Sidecar
metadata:
  name: service2
spec:
  appName: service2
  cb:
    timeout: 500
    tries: 1

---
apiVersion: proxy.company.com/v1
kind: Sidecar
metadata:
  name: service3
spec:
  appName: service3
  cb:
    timeout: 3000
    tries: 3 In this example, I want to have three CR records of type with different values. The first thing I did was specify which version of the API I wanted to use. . The CRD Declaration is an extension to the Kubernetes API, so you must specify the name of the API and course the version. You can go back and look at the line in CRD. In the spec block, I set the field values I want. Sidecar ApiVersion: proxy.company.com/v1 group: proxy.company.com kubectl apply -f services.cr.yaml
sidecar.proxy.company.com/service1 created
sidecar.proxy.company.com/service2 created
sidecar.proxy.company.com/service3 created Three objects were created. And now you can check that they are what we expect. kubectl get sidecars -o yaml                                                                    1ms
apiVersion: v1
items:
- apiVersion: proxy.company.com/v1
  kind: Sidecar
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"proxy.company.com/v1","kind":"Sidecar","metadata":{"annotations":{},"name":"service1"},"spec":{"appName":"service1","cb":{"timeout":5000,"tries":3}}}
    creationTimestamp: "2022-09-11T12:26:36Z"
    generation: 1
    name: service1
    resourceVersion: "189731"
    uid: 67f03ae5-30f6-42a5-af28-75f041c0c41d
  spec:
    appName: service1
    cb:
      timeout: 5000
      tries: 3
- apiVersion: proxy.company.com/v1
  kind: Sidecar
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"proxy.company.com/v1","kind":"Sidecar","metadata":{"annotations":{},"name":"service2"},"spec":{"appName":"service2","cb":{"timeout":500,"tries":1}}}
    creationTimestamp: "2022-09-11T12:26:36Z"
    generation: 1
    name: service2
    resourceVersion: "189732"
    uid: b874770a-257e-4044-bfb8-7ea291cdfbc8
  spec:
    appName: service2
    cb:
      timeout: 500
      tries: 1
- apiVersion: proxy.company.com/v1
  kind: Sidecar
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"proxy.company.com/v1","kind":"Sidecar","metadata":{"annotations":{},"name":"service3"},"spec":{"appName":"service3","cb":{"timeout":3000,"tries":3}}}
    creationTimestamp: "2022-09-11T12:26:36Z"
    generation: 1
    name: service3
    resourceVersion: "189733"
    uid: 934b06fb-6851-4df6-beb5-b7279d0b59b4
  spec:
    appName: service3
    cb:
      timeout: 3000
      tries: 3
kind: List
metadata:
  resourceVersion: ""
  selfLink: "" I can now manage the CR with the usual commands. For example, I can first delete kubectl delete sidecar service1                                                                
sidecar.proxy.company.com "service1" deleted
kubectl get sidecars                                                                           
NAME       AGE
service2   5m4s
service3   5m4s Then apply the declaration again kubectl apply -f services.cr.yaml                                                               
sidecar.proxy.company.com/service1 created
sidecar.proxy.company.com/service2 unchanged
sidecar.proxy.company.com/service3 unchanged
kubectl get sidecars                                                                           
NAME       AGE
service1   46s
service2   6m49s
service3   6m49s How Can I Apply it In the diagram, I showed how the architecture works, which consists of three components. Sidecar This is a container in the pod that accepts all requests and proxies to localhost, where the request should already be received directly by the service that runs in production. Sidecar is needed to implement the request processing policy. In this example, it is necessary to implement the pattern of Circuit Breaker, about which I have written above. As a sidecar, I will use Envoy, which can be easily configured via the xDS protocol. Custom Resources Kubernetes API. The database in which the settings of sidecars are specified. Yes, you can store this data in any other storage, such as Redis. But why do it when Kubernetes already has distributed storage, based on . In addition, it provides a convenient API for work, as well as the ability to subscribe to resource creation, modification, or removal events. etcd Controller This is an application that interacts with Kubernetes API and Sidecar. It may be an application in different programming languages, but in this series, there will be examples on Go, as it is for Go already have well-developed libraries. Conclusion In this part, we have looked at the general architecture of the solution, as well as at the Custom Resource Definition. In the next part, I will describe the implementation of the controller. More precisely, the part that can subscribe to events on change custom resources and respond to them. If you like my articles please encourage me in the . Noonies

Using the Kubernetes Controller for Envoy

Migrating from bare-bones Envoy to Istio This post is part of the “Service Mesh” series. If you haven’t read the previous posts, I would urge you to do so, it will help understand this article better. Here are the previous articles Service Mesh with Envoy 101 Microservices monitoring with Envoy Service Mesh, Prometheus & Grafana Distributed Tracing with Envoy Service Mesh & Jaeger In the previous articles we looked at installing Envoy as a side car proxy to our services and saw how the Service Mesh setup helped in managing traffic between the services and to collect and visualise tons of telemetry data about the traffic. There were a few annoying things in the setup though Installing the Envoy side car manually Routing traffic to the side car proxy manually Managing the configurations for all the Envoy side cars Regarding #3, as we saw already in one of the previous posts, we can use Envoy’s xDS server to manage the configuration in a central way and update them dynamically without needing a restart. Yes, this reduces the pain to a level, but still you have to maintain the configuration for all the side car proxies. Data Plane vs Control Plane Before improving our setup, we need to understand these terms which are widely used in a service mesh context. Data Plane is the one which actually does all the ground work like routing traffic, collecting telemetry and sending to the metrics store(e.g.: statsd), managing traffic with circuit breaking, rate limiting, etc.. An example of Data Plane would be our Envoy side car. Control Plane is the one which configures the side car proxies in the service mesh. Till now and configuring the side car proxies, but we can use a tool which will act as a Control Plane and configure our side car proxies for us. Example of a Control Plane is Istio. we were acting as a Control Plane manually Istio Control Plane So now you don’t directly configure any side car proxies, we submit our configurations(traffic shifting, fault injection, etc.. )to the config store, Istio Pilot(a component in Istio) looks for changes in the config store and then pushes these changes to the side car proxies. Note: The above diagram shows only Istio Pilot, but Istio has several other components like Citadel, Galley, etc… Demo Let’s look at an example of setting up a Service Mesh with Istio. We will deploy our services in a Kubernetes cluster Service Architecture Installing Istio Pre-requisites: You need to have a Kubernetes cluster up and running Have Helm client and tiller configured in your cluster. Refer here Clone the official Istio repo kubectl apply -f install/kubernetes/helm/istio/templates/crds.yaml helm install install/kubernetes/helm/istio -name istio -namespace istio-system Note: Am using Helm version v2.9.1 and the Kubernetes cluster is in GKE. You can find installation instruction for other environments . here Deploying the Services When we were dealing with only Envoy, we had to manually route the requests to the side car. This time we would not do that, we would let Istio do the magic. The source code of the services can be found . Nothing special, just a service calling a couple of other services. Also there is no Envoy configuration for each service, Istio will take care of the side car configurations. Service A while calling Service B and Service C, uses the Kubernetes service name, “serviceb.serviceb” and “servicec.servicec” respectively here We are creating Kubernetes namespaces for our services and we are adding a special label to each namespace, we will discuss why that is necessary shortly. We are managing our applications with helm and you can find the helm charts for the applications in the same under the “helm” directory . If everything went well, your pods should be up and running repository service pods running If you watch closely there are two containers(in each pod) running for every service, but in our helm chart definition we had only one container defined. This happened because Istio watches over all the deployments and adds the side car container to our pods.This is achieved by leveraging what is called , this feature was introduced in Kubernetes 1.9. So before the resources get created, the web hook intercepts the requests, checks if “Istio injection” is enabled for that namespace, and then adds the side car container to the pod. So that is how Istio solves the problem of manually adding a side car proxy to each of our services. MutatingAdmissionWebhooks Note: Namespace labels is not the only way to add the side car, you can even use the “istioctl” command line tool to generate the specification and deploy Next we create a Ingress rule to route traffic to Service A. and are Kubernets CRDs(Custom Resource Defnitions) created when we installed Istio. We will see more about these in the next post, for now just understand that we are asking all requests to be routed to Service A. Gateway VirtualService Let’s hit the gateway’s public ip to generate some traffic. You can find the external ip of your gateway by kubectl get services istio-ingressgateway -n istio-system We can use any load testing tool to generate some traffic, am using hey hey -z 30s http://x.x.x.x Istio comes with tools to monitor what is going on in the service mesh, let us look at Kiali and Grafana. Note: When we installed Istio with Helm, tools like Grafana, Prometheus, Zipkin, Kiali, etc.. would have been automatically installed. You can control what is installed and their configurations by modifying the values . file Kiali kubectl port-forward service/kiali -n istio-system 20001:20001 Kiali service mesh visualisation Grafana kubectl port-forward service/grafana -n istio-system 3000:3000 Grafana metrics As you can see, we have our Service Mesh with Istio setup and we have all the metrics and visualisations. But a thing to note is . Istio uses IP Table rules to route incoming and outgoing traffic to side car proxies first. More details about the rules . we did not manually route traffic to the side car proxies, it automagically happened here Configuring Istio We saw that you don’t directly configure the side car proxies with Isito. So how do we configure traffic control rules? For this purpose when you install Istio a lot of Kubernetes CRDs (custom resource definitions) are created, E.g: , … We see an example of a configuration to controlling traffic below: VirtualService DestinationRule The above rule is an example of doing canary deployment, routing 10% traffic to the new version of the service. So we submit these rules to the config store, Istio Pilot watches the store and pushes the configuration changes to the appropriate side car proxies. Conclusion With Istio, there is literally no code change that the service developer has to do to get all the benefits of a Service Mesh. They can instead concentrate on building business features. In fact, the developer wouldn’t even know that there is a Service Mesh setup. This post was just to see how migrating to Istio solves some of the annoying issues we had with Envoy. And that is why we didn’t delve deeper into Istio. Istio is much bigger and has many more things to it. In the next post we will explore more about Istio. You can find all the code . I gave a talk on this topic which you can find here here _Contribute to dnivra26/istio_101 development by creating an account on GitHub._github.com dnivra26/istio_101

Service Mesh: Moving from bare-bones Envoy to Istio

In this article we will briefly discuss about what this “Service Mesh” buzz is all about and how we can build one using “Envoy” WTH is a Service Mesh? Service Mesh is the communication layer in your microservice setup. All the requests to and from each one of your services will go through the mesh. Each service will have its own proxy service and all these proxy services together form the “Service Mesh”. So if a service wants to call another service, it doesn’t call the destination service directly, it routes the request first to the local proxy and the proxy routes it to the destination service. Essentially your service instance doesn’t have any idea about the outside world and is only aware about the local proxy. services with local proxies When you talk about “Service Mesh”, you will definitely hear the term “Sidecar”, a “Sidecar” is a proxy which is available for each instance of your service, each “Sidecar” takes care of one instance of one service. sidecar pattern What does a Service Mesh provide? Service Discovery Observability (metrics) Rate Limiting Circuit Breaking Traffic Shifting Load Balancing Authentication and Authorisation Distributed Tracing Envoy Envoy is a high performant proxy written in C++. It is not mandatory to use Envoy to build your “Service Mesh”, you could use other proxies like Nginx, Traefik, etc… But for this post we will continue with Envoy. Okay, Let’s build a “Service Mesh” setup with 3 services. This is what we are trying to build services setup with sidecar proxies Front Envoy “Front Envoy” is the edge proxy in our setup where you would usually do TLS termination, authentication, generate request headers, etc… Let us look at the “Front Envoy” configuration Envoy configuration majorly consists of Listeners Routes Clusters Endpoints Let’s go through each one of them Listeners You can have one or more listeners running in a single Envoy instance. Lines 9–36, you mention the address and port of the current listener, each listener can also have one or more network filters. It is with these filters that you achieve most of the things like routing, tls termination, traffic shifting, etc… “envoy.http_connection_manager” is one of the inbuilt filters that we are using here, apart from this envoy has several other . filters Routes Lines 22–34 configures the route specification for our filter, from which domains you should accept the requests and a route matcher which matches against each request and sends the request to the appropriate cluster. Clusters Clusters are the specifications for upstream services to which Envoy routes traffic to. Lines 41–50, defines “Service A” which is the only upstream to which “Front Envoy” will talk to. “connect_timeout” is the time limit to get a connection to the upstream service before returning a 503. Usually there will be more than one instance of “Service A”, and envoy supports to route traffic. Here we are using a simple round robin. multiple load balancing algorithms Endpoints “hosts” specify the instances of Service A to which we want to route traffic to, in our case we have only one. If you note, line 48, as we discussed we do not talk to “Service A” directly, we talk to an instance of Service A’s Envoy proxy, which will then route it the local Service A instance. You could also mention a service name which will return all the instances of Service A, like a headless service in kubernetes. Yes we are doing client side load balancing here. Envoy caches all the hosts of “Service A”, and for every 5 seconds it will keep refreshing the hosts list. Envoy supports both active and passive health checking. If you want to make it active, you configure health checks in the cluster configuration. Others Lines 2–7, configures the admin server which can be used to view configurations, change log levels, view stats, etc… Line 8, “static_resources”, means we are loading all the configurations manually, we could also do it dynamically and we will look at how do it later in this post. There is much more to the configurations than what we have seen, our aim is to not go through all the possible configurations, but to have minimal configuration to get started. Service A Here is the Envoy configuration for “Service A” Lines 11–39, defines a listener for routing traffic to the actual “Service A” instance, you can find the respective cluster definition for service_a instance on 103–111. “Service A”, talks to “Service B” and “Service C”, so we have two more listeners and clusters respectively. Here we have separate listeners for each of our upstream (localhost, Service B & Service C), the other way would be to have a single listener and route based on the url or headers to any of the upstreams. Service B & Service C Service B and Service C are at the leaf level and do not talk to any other upstreams apart from local host service instance. So the configuration is going to be simple so nothing special here, just a single listener and a single cluster. We are done with all the configurations, we could deploy this setup to kubernetes or use a docker-compose to test it. Run and hit , you should see the request pass through all the services and Envoy proxies successfully. You can use the logs to verify. docker-compose build and docker-compose up localhost:8080 Envoy xDS We achieved all of these by providing configurations to each of the side car, and depending on the service, the configuration varied between the services. It might seem okay to hand craft and manually manage these side car configurations initially with 2 or 3 services, but when the number of services grow, it becomes difficult. Also when a side car configuration changes you have to restart the Envoy instance for the changes to take effect. As mentioned earlier we can completely avoid the manual configuration and load all the components, Clusters(CDS), Endpoints(EDS), Listeners(LDS) & Routes(RDS) using an api server. So each side car will talk to the api server to get the configurations and when a new configuration is updated in the api server, it automatically gets reflected in the envoy instances, thus avoiding a restart. More about dynamic configurations and is an example xDS server which you can use. here here Kubernetes In this section let us look at, if we were to implement this setup in Kubernetes what would it look like single service with envoy side car So changes are needed in Pod Service Pod Usually Pod spec has only one container defined inside it. But yeah as per definition a Pod can hold one or more containers. Since we want to run a side car proxy with each of our service instance we will add the Envoy container to every pod. So for communicating to the outside world, the service container will talk to the Envoy container over localhost. This is how a deployment file would look like If you see the containers section, we have added our Envoy side car there. And we are mounting our Envoy configuration file from configmap in lines 33–39. Service Kubernetes services take care of maintaining the list of Pod endpoints it can route traffic to. And usually kube-proxy does the load balancing between these pod endpoints. But in our case if you remember, we are doing client side load balancing, so we do not want kube-proxy to load balance, we want to get the list of Pod endpoints and load balance it ourselves. For this we can use a “headless service”, which will just return the list of endpoints. Here is what it would look like line 6 makes the service headless. Also you should note that we are not mapping the kubernetes service port to app’s service port but we are mapping it Envoy listeners port. Traffic goes to Envoy first. With that you should be good to go with kubernetes as well. That’t it. Looking forward for your comments. This article is kind of a pre-requisite for the articles , & . Please go through them if you are interested. Distributed Tracing with Envoy Monitoring with Envoy, Prometheus & Grafana Migrating from Envoy to Istio You can find all the configurations and code . here _Contribute to dnivra26/envoy_servicemesh development by creating an account on GitHub._github.com dnivra26/envoy_servicemesh

Service Mesh with Envoy 101

If you are new to “Service Mesh” and “Envoy”, i have a post explaining both of them . here This is the second post in the series, you can read the first post about Distributed Tracing . Observability with Envoy service mesh here With microservices you cannot be in the dark when it comes to monitoring, you need to at least know that something is going wrong. Let us look into how Envoy can help us get some info on what is going on with our services. With a service mesh, all the traffic goes through the mesh, meaning no service talks to the other service directly, the service make a call to Envoy and Envoy will route the call to the destination service, so Envoy will have context about the incoming and outgoing traffic. Envoy generally provides metrics about the requests, requests and the . incoming outgoing state of the Envoy instance Setup Here is an overview of what we are trying to build overall setup Statsd Envoy supports publishing metrics in 2 or 3 formats, but for this post we will use format. statsd So with that said, the flow will be, Envoy pushes the metrics to statsd and from statsd we will pull the metrics using (a time series database) and then we will visualise the metrics using . prometheus grafana In our setup diagram i have mentioned statsd exporter instead of statsd for a reason, we are not going to have statsd as such, we are going to have a converter(service) which will accept data in statsd format and expose it in prometheus format. Gets the job done for us. Envoy’s metrics can be majorly classified into two Counter: An ever increasing metric. E.g.: total number of requests Gauge: A metric that can go up or down, like an instant value. E.g.: current CPU utilisation Let us look at an Envoy configuration with stats sink lines 8–13 tells Envoy that we need metrics in statsd format, what is the prefix for our stats(usually your service name) and the location where our statsd sink lives lines 55–63 configures the statsd sink in our environment that is all the configuration that is needed to get stats out of Envoy. Now if you look at lines 2–7, there are two things happening Envoy exposes an admin endpoint on port 9901 which you can use to dynamically change the log level, view current configuration, stats, etc.. Envoy can also generate access logs similar to nginx, which you can use to understand your traffic. the format of the access log is also configurable, lines 29–33 does exactly that You need to add the same stats configuration to the other side car Envoy’s of the services in our system (yes, every service has its own Envoy side car). The services themselves are written in go and they do not do much except for calling other services through Envoy. You can look at the service and Envoy configurations . here So right now we only have statsd exporter in the picture, with this, if we run the docker containers(docker-compose build & docker-compose up) and send some traffic to Front Envoy(localhost:8080), Envoy would start sending metrics about the traffic to our statsd exporter, which will convert the metrics to prometheus format and expose it in port 9102. This is how the stats look like in statsd exporter metrics from statsd exporter in prometheus format there would be hundreds of stats and in the above screenshot we are seeing the latency metrics for communication between Service A and Service B. The metrics in the above image are in prometheus format metric_name ["{" label_name "=" `"` label_value `"` { "," label_name "=" `"` label_value `"` } [ "," ] "}"] value [ timestamp ] You can read more about it . here Prometheus We are going to use as our time series database to store our metrics. Prometheus is not just a time series database, it is a monitoring system in itself, but in our setup we will use it as a datastore for our metrics. An important thing to note is prometheus is a pull based system, which means you have to tell prometheus where to scrape the metrics from, in our case it will be our statsd exporter. Prometheus Adding prometheus to the equation is very straightforward, we just need to pass the scrape targets(statsd exporter) as a configuration file to prometheus. Here is what the configuration will look like scrape_interval is the frequency in which prometheus will pull configuration from the target. So now we should have Prometheus up, and some data in prometheus as well. Let’s fire up locahost:9090 and see what it has prometheus query page as we can see, our metrics is available. You can do a lot more than just selecting existing metrics, you can read about prometheus query language . It can also plot graphs based on our queries. Has also an alerting system. here If we load up the targets page in prometheus we see all the scraping targets and health of those targets prometheus targets Grafana Grafana is an awesome Visualisation & Monitoring solution which supports a lot of backends like Prometheus, Graphite, InfluxDB, ElasticSearch, etc... Grafana has two major components that we need to configure Datasource: The backend from which grafana will get the metrics. You could configure the datasource using a configuration file which will look like this 2. Dashboard: This is where you visualise the metrics from your data source. Grafana supports a wide variety of visual elements like Graphs, Single Stats, Heatmaps, etc… and you can extend this and build your own using plugins. The only problem i have with Grafana is that there is no standard way of developing these dashboards as code. There are some third party libraries which support this and we will use the one from weaveworks called . grafanalib Here is the dashboard that we are trying to build expressed as python code We are building graphs for 2xx, 5xx and latency. lines 5–22 is important, it is extracting the service names available in our setup as grafana variables, it makes our dashboard dynamic, meaning we will be able to select the source and destination service for which we want to view these statistics. More about variables . here You have to use the grafanalib command to generate the dashboard from the above python file generate-dashboard -o dashboard.json service-dashboard.py beware the generated dashboard.json is not easy to read. So we just need to pass the dashboard and the datasource while starting up Grafana. And when you visit http:localhost:3000, you will be greeted with: grafana dashboard there you go, you have your 2xx, 5xx and latency charts and you also see the dropdown where you can select the source and destination services. There is more to grafana than what we have discussed, there is a powerful query editor, an alert system. More importantly, everything is extensible using plugins and applications, checkout an example . If you are visualising metrics of common services like redis, rabbitmq, etc.. Grafana has a repository of from which you can just import them and use. One more good thing about Grafana is you can create and manage everything with configuration files and code without dabbling much with the UI. here public dashboards I would urge you to play with prometheus and grafana to figure out more. Thanks for you time. Please leave your feedback as comments. You can find all the code, configuration files . here _Demo for monitoring micro services with envoy service mesh, prometheus & grafana - dnivra26/envoy_monitoring_github.com dnivra26/envoy_monitoring

Stories, News & Mentions about ENVOY

Envoy’s Stories on HackerNoon