The HackerNoon Podcast: Managing Databases on Kubernetes with Anil Kumar

Listen to The HackerNoon Podcast on Apple Podcasts , Spotify , or wherever you listen to your podcasts . Amy Tom talks to Anil Kumar, the Product Manager at Couchbase, about becoming a Product Manager, writing his book: Couchbase on Kubernetes, and creating a multi-cloud strategy. 😁 This week on The Hacker Noon Podcast: Anil has been at Couchbase for 8 years, and prior to that was at Microsoft for 7 years. Amy needs to know - what is it like to grow within a company? 🤯 (05:25) Anil did his Bachelor's and Master's in Computer Science. Amy asks if he has any regrets about his education experience (10:10) Anil talks about the book he wrote: Couchbase on Kubernetes 📗 (12:03) Anil talks about what you need to manage the database and the levels of complexity with databases and multi-cloud strategies (16:00) What are the trends around Kubenetes containers and databases? 📈 (19:42) Anil gives advice to businesses looking to move their database operations to Kubernetes (27:15) Follow Anil Kumar and Couchbase: You can follow Anil on Twitter @anilkumar1129 You can connect with Anil on LinkedIn: https://www.linkedin.com/in/anilkumar29/ You can follow Couchbase on Twitter @couchbase Shownotes: Check out Anil's book: Couchbase on Kubernetes on Amazon: https://www.amazon.com/Couchbase-Kubernetes-Autonomously-Distributed-Database/dp/0692039724 Become a Hacker Noon contributor https://app.hackernoon.com/new Podcast Transcript (Machine-Generated, Please Excuse The Errors) Amy: [00:00:00] Amazon prime day is coming up. I know that we don't necessarily love Amazon because Jeff Bezos, but I know Amazon prime day is coming up and I just looked up. Amazon prime in Canada. And unfortunately Amazon prime day in Canada is canceled or slash per postponed, due to COVID. And I feel like a child who's gotten their candy taken away from them or something. I feel like I'm being reprimanded. Not into it, don't like it. So please everybody go get our vaccines so that we can have Amazon prime day. Anyways, of course, what's up, this is the hacker noon podcast and my name is Amy Tom. Today. I am joined with Anil Kumar, who is the director of project management at Couchbase. How are you doing Anil? Anil: [00:00:50] Hey I'm doing good. Amy, and thanks for having me on your show. Amy: [00:00:54] He is today. We are going to get into some information about Couchbase and Kubernetes and also cloud strategy. But before we get into that, I want to ask you a few questions about you. How long have you been the project manager at Couchbase? Anil: [00:01:11] Yeah. Yeah. So maybe sorry, I'll just make one correction. So it's product management, but yeah, so I've been in, so I've been with the coach base for eight plus years. And then I joined coach base. We were maybe 30, 30, 35 people in the company. And today more than 600 people. Yeah. So how long ago was that? It was eight years back, a tiny company in mountain view, California. And it was for me was a big change. And before coming to coach base, I was in Microsoft and we all know Microsoft, is it. Is a big company and even a smallest team in Microsoft is in the size off that 60 or 70 people. And so it was a big change for me to giant and big company probably become midway. Small has been amazing. Got to learn lot, many things and help shape the product and multiple ways. My my, my daughter done in college. Amy: [00:02:07] So did you say 35 number 35 years? Anil: [00:02:10] Yeah, not specifically, but Amy: [00:02:14] so now are there other people that you worked with from the beginning that are still at Couchbase as well? Anil: [00:02:21] Yeah, we still have a lot of folks that makes me actually very happy because the, especially the core engineering team, we started building from the beginning. Most of those guys are still there. And so I'm proud and happy that we all share the same goal and same passion for preliminary plus class database. And so that's actually, it makes me even that. Stick around here for some more long time, because it's technology, it's all the people in the family that created it Amy: [00:02:51] eight years is a long time for one company. Yeah. The longest I have worked at one company was almost three years and by the end of it, I was quite honestly bored. So I don't what made you, what motivated you to continue and to grow within Couchbase. Yeah. Anil: [00:03:12] And by the way, this was not the first time. So I worked in Microsoft for seven plus a year, so Amy: [00:03:19] that's okay. You're a long-term or I see, Anil: [00:03:23] but mainly it was like the with the coach base. I joined although I joined as a product manager, but I was put on to doing multiple. I was changing my hats every second in my thing, but it was like, I did the training. I laid it all off solution architect. And then I also helped with the help our CEO who's on legal stuff. So I was doing multitasking, multitasking, billable things. Which was great experience. One of the reasons why a giant touch base was mainly to get that kind of experience and exposure. And it was the right fit and the, yeah but from there on, we obviously started greeting deem and we have no full fledged legal team, full fledged partner being a solution architect thing. But I can still always go back and say, Hey, I do remember the early days both for those customers. I was the solution architect. We shouldn't, we have been doing is amazing in my opinion. The world-class team and the tear differentiation is themselves product. Every feature we think through and make sure that these are uniquely, both and clearly differentiates the market. And one of the thing which I'm passionate about is the customer opposition we have in the company. So it's all about customer or the challenges with the customer. How do we solution. Oh, is it going to jobs? That's all the customer Amy: [00:04:38] challenge. And do you talk to the customers directly as the product manager? Yeah, Anil: [00:04:44] a hundred percent of the time, I would say have a great relationship with our customers and they trust me and nothing that makes me very happy. And whenever I see a tweet or a blog that put a blog or post something on the LinkedIn. I can certainly say most of the customers do respond and say the the, congratulate me or because Amy: [00:05:05] they know you because you've been there for eight years. How many people do you manage? Anil: [00:05:12] So at this point, I have only one person managing. At one point I had a team of three or four people, but as we transition there we, when we see the one of the things which we have done also in codebase is we increase people to move at all. If they'd like to explore another area, And in some cases we say, Hey, you're a great fit for this team and you have some great ideas and you'll be valuable in this team. And so we do punish in that. Amy: [00:05:37] Okay. Cause I'm just wondering, I, maybe this is a selfish question for me, but with hacker noon, I started working at hacker noon, maybe. Six seven months ago at this point. And we have grown the team almost two X since I've been here. And so my question to you is how did you grow within this team? When so many new employees and new people were coming when you're used to wearing so many hats, and now you are maybe a little more siloed, Anil: [00:06:06] Yeah, not truly. I think at one point I would remember everybody's names. So it was like, you told another guy. I know this guy, I don't need us, but yeah, no, the team is bigger and it's not possible to know everyone in the company, but it's one way we have at least the. The idea for me, it was like we have the product team concept of each of those products we built there, the product team, and this is the core productive, right? So everything includes engineering to me, product management, project management, and then also support. So it's. It becomes a theme, which is solving this customer problem. And there's no track, so there's no ambiguity or there's no no, the focus is very clear. It is. And everyone is aligned on the same goal. And so this product team concept is what I really love. And then for the multiple products, which I manage, I work on this productive concept. Amy: [00:07:01] Okay. Yeah. So let's talk about the actual work that you do, the product that you're managing, or I guess like really what it means to be a product manager. I feel that's really it's a vague title. So what are you managing? Tell me more about that. Anil: [00:07:16] So for a manager is more of like the I would say there's different times that product owner or someone who's, we call it as an entrepreneurial or like the CEO of the product. So there are many times we use they are truly enablers, right? So it's the, when you think about from the concept. She'll have the idea that from the idea to execution and being new to the market and making it successful. And so there is a, there's a, the entire life cycle and the product manager's role. It's a tough job, to be honest with you. It's like you have to have a, you need to have a lot of research and understanding of the industry, customer pain points, and really deeply understand those. And then stop. Building the product, right? So you have to have a business justification of see how this is aligned with the company's vision. And also think about the, how this product is going to grow. So that, and then again, convincing engineering is not easy. Obviously our engineers are the smartest engineers on making sure that we are utilizing their time and resources. And for the right product and the right the capabilities and taking through that execution, getting it out of the door and then making it successful, which is where you'll be working with the field and support and savings to talk to the customers and adopt, and then come back with the feedback. So it's a, full-time like truly a correct order. Amy: [00:08:43] So would you say you're like the bridge between development and business side? Action Anil: [00:08:48] speaking, if you a bridge for entities from paid from the customer engineering field sales, legal, you name it like pretty much your bridge for everything, Amy: [00:08:59] Okay. Then how do you not let people walk all over you? Anil: [00:09:03] Yeah, it's a, as I said that you have to so once you. Start working as a product manager, you will learn on the job, but you will start learning those skills. It's the people, management is very important. Domain expertise is also really important. So you need to know the domain really well. And then obviously as I said you'd be working with the techno broad spectrum of all the people. So you need to know what they expect and who are the stakeholders, what their what their expectation is. So you have to it's as I started working on this, on product management and started getting those skills Amy: [00:09:36] would you say that you're a natural people person? Anil: [00:09:38] I think so. Amy: [00:09:39] Yeah, because I think that to do that job, especially to manage all those shareholders and moving parts, you need to be really good at talking to people. Yeah. Anil: [00:09:49] Talking to people, building relationship is Amy: [00:09:52] important. Yeah. Yeah. Especially with the customers, like you were talking about engaging with them on LinkedIn and things like that. Building engagement and relationship with the customers. I feel that it would be more challenging to do that if you were maybe like more introverted or less people focused, maybe more like a developer mindset kind of deal. What was your educational background in. Anil: [00:10:12] I did my master's and a bachelor's both in computer science. Amy: [00:10:16] Okay. Okay. So it wasn't in like a business realm. It really was in like the technical side of it. Anil: [00:10:21] Yes. Before becoming a product manager that transition happened in Microsoft. I was actually an engineer, like a software engineer. And, but mainly my domain expertise was on the database and data application and scalability and all of that. And then from there, I transitioned to the program manager, the role in Microsoft and from there when I joined codebase, it was Birdman. Amy: [00:10:42] Okay. So do you think that you, if you could go back and do it again, would you have gone into more of the business route first, then get into technical? Or are you happy that you did the technical and then went more into learning about business? Yeah, no, Anil: [00:10:55] I, I, yeah. I think for any, it is important to have that foundation. And I feel like it was the right thing to do for me, was to learn the foundation very well back. So it was if I'm going to build a product and if I so it was important to understand that maybe to become an engineer, you would know how to build the product. And then from there you start looking at it. Are we solving the right set of problems for the customer and what's the business sign up rate. And so then we move into the business side effect. So that's, in my opinion, I think that was a good position. Amy: [00:11:28] Yeah. Makes sense. Cool. Okay. So I want to ask you about Kubernetes, about resource consumption about cloud portability. How did you learn all of that on the job? Anil: [00:11:39] I did learn on the job and the main thing about in fact one of the things which I've done in one sec, so yeah, so I learned it on the job. And it was mainly that this was a engage technology back in 2016 and 20 16, 20 17. And I was very interested in learning about this technology because There was definitely a hype cycle, which was like with containers and Docker and stuff, but for what it is, was emerging. And no, I think there was a increasing like the, this open source project was getting a lot of interest from the developers. And I started learning about, and in fact, I became so interested that when we launched a product in 2018, I don't even book on the covenant. So yeah. So yeah, so it was good complementary to the product launch because I wonder if to make the launch successful. The idea was that it was like the the coach based on Kubernetes is a first in the industry to build a complex distributed database running on it. But it is technology. And so I think, as I said, I was very much interested in this, in Amy: [00:12:46] this technologies. Okay. So you're saying like back in 20 17, 20 16, it was common to make a clump complex system based on Kubernetes structure. What is it? What is the trend moving towards now? Anil: [00:12:59] So the, there was a, in this case, let me tell you a bit of a small short story. I think it will be important set the context here. So if I If I look back and 20 13, 20 14, and in those years there was, as I said, like there was a hype cycle with the containers and Docker containers, especially now those conferences were, had a attendance, often thousands and stuff, but so you can see there was a lot of interest from the developers who wanted to adopt this technology. And those were really adept fit for state less applications. When I say state less, these are your applications, which doesn't still have any persistence data. I tell you, they don't, there's no storage for those, but I think more database, it's a stateful application, which you need to store the data onto the desk. And so there was initial hindrance or like kind of step dissolvable, like adopting. Containers for database. And so this is, as I said, back in 20 13, 20 14 there you'll see a lot of articles were published and some really some of the industry, people, which are the K is database, a good fit for containers. We should not run containers, database monitor containers. There were those kind of articles published. Yeah. Amy: [00:14:14] And that was like 2013, then Anil: [00:14:17] 2014 until 2016. Fast forward 2019, the same folks or your, like the other folks started writing the articles. Yes, they do have is a good fit for containers, but maybe not for production. And then now we're pretty much the database are mainstream, mainstream on containers. And why? Because yes, they, the maturity of the technology itself and the, in terms of the the what we had our level at the time. Yes. There was a lot of those. It required the maturity and the all the stuff to come together. So our journey in this case was like, we in fact started baby ahead of the other databases our co-founder of the company and Dustin selling At the time he was the highest, but one of the highest the cash product, he in fact, was doing a experiment of running touch based on Bucko container. Eventually docketed soak was in the alpha state in the points. Okay. It goes back to 2013 and then 2016, we took a. One step ahead. And then we launched co-space on containers as a production deployment, right? So it was ready for the production. Most of the customers did not want it to go run that much based on containers, but our goal was like, we want her to stand as a thought leaders in the industry. And from 2016 to 2018, we launched the post based on Kubernetes. And now if you look, if I look at we've been praised as one of the thought leaders in the database for adopting the comparison we have this service has become a fastest growing servicing cost base, and there's a huge excitement in the customer. So it's, so you can see that it rolls a vision, which we had from a few years back, and we have been a fight to work on that vision and become a cloud native data. Amy: [00:16:01] I guess why Kubernetes though, because I know that I talked to Matt about this a little bit as well in one of our previous episodes, but why would you use Kubernetes? Because it's complex, right? Anil: [00:16:14] Yeah. Yeah. That's a good question. Let me first stop using the commodity computing. There, there are many overheads associated with managing the data center, power and cooling maintaining the networks, which is unwelcome. And then the servers and storage dental were more complex. So what changed was the automated orchestration started happening and that made more efficient, but in the end it was the managed cloud. That was the real game changer, in my opinion. So the physical resources replaced by what you know, and to be created with the click of a button. And so the benefits were better resource utilization and vastly improved the time to market. It would be the traditional procurement and leading the cost savings and stuff. So how long would we watch on machines? They still needed the management. They did it, a security policies, auditing and monitoring. What normally comes coming into the Kubernetes. It's the next giant leap forward, right? Delegating this requirements to the cloud operator, living the business, to concentrate on creating and deploying their applications. I think that's where the Kubernetes has become a last few years. It has become a big giant leap forward in terms of technology, Amy: [00:17:28] but that's where I want to layer on the aspect of like cloud portability cloud strategy too. Does it not get even more complicated in that instance, if I am using like a hybrid cloud or cross cloud strategy? Anil: [00:17:40] Yeah, no, that's a good question. In my opinion, the Kubernetes maybe complex, but it is not unnecessarily complex, right? So what I mean by that is in fact, the complexity of container orchestration has led to the advancement of Kubernetes as a leader in simplifying and deploying. And management of the containers. So nobody's speaking whatever complexity, the is attributed to Kubernetes is significantly lower than what we find when attempting to deploy application X K with the plane or Docker or containers. So solving those problems in biplane, but coming to the question of portability. But it came into the existence. They put on it as it was considered as a club, the container orchestration, but it has evolved much more. It has become a quote agnostic infrastructure and Gnostic solution. It can be highly extensible, so you can extend and build your application on the Kubernetes. And, but mainly customers lo the covenant is mainly for oven. When you think about when you built a application on Kubernetes no, they the CNCF, which is the cloud native computing foundation has this certified Kubernetes platform and every vendor who that is vendor, who wants to be posed to have a Kubernetes service, has to go through the certification. And so that certification mix shows that the are compatible. They are they all the APS, which are. Are we able to, the APA are certified on those platforms? So what that means is that if I build my application today, I should be able to afford this upon the open source Kubernetes, to run on any of the managed Kubernetes service. And it should work fine with. So it should not take much because as I said, the APS are exactly the same, right? Yes. Each cloud has their own although they do add some more capabilities on top of it, they have a different policies, foreign security, like the some cloud. Big the security parts as much often than others, but the idea is like, you shouldn't be buried once and run anyway. It's Amy: [00:19:40] okay. I wanted to ask you about the companies that operate on this kind of structure and system. And I guess I want to know specifically yeah. The trend towards moving to the system. And that just the fact that you cement Jen, that this is the now like the most trending product that Couchbase has right. Or service what were people on before moving to this and why would they have moved? You know what I mean? I want to know what is a common structure alternative to this, that this would be better. Anil: [00:20:13] Okay, sure. I think that's a big question. Before, before they this whole containers and Kubernetes they typically customers would deploy coach base. They're very much performance conscious. They will be point on the bandwidth on the physical machine so that they can get the highest and best performance. But there are costs looking at optimizing the cost and they have multiple applications running. So they will be part on a virtualized environment, like on VM instances. So in both the cases so they would typically a committee. I don't do those database operations. They were write scripts, write those best practices, specs to manage the eco twist cluster provisioning scaling maintain and upgrade and office operations. So they will break those clips. And so they would maintain those scripts. I told there would be. So you'd see that deputy what we seen in the, in our customer basis. They will become a coach base champions. There'll be one or two or three, four folks will become sports-based champions who knows in the note, they will start building those scripts, the scripts themselves, and they can use chef Ansible, puppet or whatever those tools they have to basically automate those operations. And that was the trend, or that was the case for last many years. And what has changed now is because the, in this case, like this possible champions, now they have to build those scripts, maintain those. And then if there is a new version comes, they have to go and again update those scripts. So there's a lot of activity which they have to do. With the, with continuous improvement is what happens is we embedded the best practices within the campaigns itself and that they are autonomous operator for Kubernetes. Now we have automated all our best practices. So basically we are taking away that, that job, which the DevOps or the engineering database engineer was doing, they're taking away their job and we are simply finding it. So the exports of coach based who built the coach based product, they have they throw out those best practices, guidelines, and implementing those best practice guidelines. So these are some really complex opera operations we do. For example, if we have a hundred and old cost-based cluster and you want to upgrade this cluster and this isn't redundant machine critical applications, we want to make sure there's no downtime. And if there's a, if there's any gift, just how do we handle those? So all of that the customer can handle this before now with the operator. And the automated, the whole operation, all those educates scenarios. And without all expert with our experience working with Justin was we have endured those best practices, diamonds, in my opinion, it's it reduces the operational costs significantly. So we have got quotes from customers that it has reduced the operational costs by 90 to 95%. What Amy: [00:23:02] kind of industries are these customers in? They, Anil: [00:23:05] they this span across multiple industries. We have customers in the travel and the retail as well as in financial districts. And so we have multiple and again, I decided that when I say this is the fastest growing, it's been, it's another thing customer adoption has been significantly higher on. Amy: [00:23:23] Okay. And I guess okay, maybe this is a dumb question, but because I think one of the benefits of Couchbase I believe is auto scalability, but I guess when people are on the, this like Kubernetes structure, do they have to worry about how large their database becomes or how much querying they're doing? Anil: [00:23:40] So you're asking to identify this person. And again, in general, the database person will tell you like it's a, it's the domain expertise that someone who has worked in this database knows really better. Now, what are the complexities of scaling? It's not just I want to add a multiplicity and I'll show some more nodes to the cluster. Because that has its own implications. So when you add nodes to the cluster, we have to rebalance that we have to sharp our basically redistribute the data. And depending on the size of the data, we have that our weights of data sitting on each notice. Now we have to redistribute all the data across all those new notes. That's a, it's a, it lets us network. You are if we have a habit tracker coming in, you have to make sure that it doesn't impact those the incoming traffic. So there's lot of intelligence built into that. So what used to happen before is like most of our customers in this space, they will ops or they learned or job title or what stacks to look at, what metrics to look at and based on those, and they're experienced, they were basically scaled when they tell stories in not non-peak timeframe and then they will stay up the cluster. But it required a lot of intelligence, right? It's a lot of thinking about like when to stay and how to complete this whole operation what I've done. And this is where I think we are becoming more and more economics databases that we take those complex operations and fully automating them. And so we know recently is which is, which actually went live yesterday. The Causeway synonymous operator, we launched the of based services. And so giving this capability to the, those DevOps engineers or the service engineers, they can put their at those business logics, which is I want to scale my cluster when I hit some threshold, whether it could be a system threat, like CPU, memory, storage network based, whatever, or it could be that, Hey, I want to I'm anticipating that I'm going to get more traffic and I need to add more capacity. So what are the metrics that they can put those thresholds and the system they post base operator like romantically Kayla, their services. Amy: [00:25:49] How did that differ from what you had before? 2.2 Anil: [00:25:52] though. Before we scaled the cluster, it was manual operation substance. Like you would basically say, I am looking at all this metrics and I want to scale up my cluster. You come and basically make a change in your AML five, which is a configuration file. And this, when the operator sees that there's a change in the configuration, it will go and scale up the cluster, but it was not fully automated. So my apple make it in substance. So now it's once you've defined those specials and you don't, and then the rest of the things, which is like operator manage, look at those metrics. And then when those thresholds are hit, it will go and pick up the kids. Ah, Amy: [00:26:29] okay. And is it the same width? Down-scaling yes. Anil: [00:26:37] Now you can scale up on the scale down ideally for production use cases. Obviously you could tell customers it's a database. It's not a steep, less application. So when you're saying. Getting done, make sure that it doesn't impact the results and impact your performance because then you're still in down. Obviously it's basically they're using the capacity, but also there is a redistribution of the data again, so we might come back to your data. But if we are there are services in cockpits, like query, which is the the nickel service that is a stateless and yes, you can scale up and scale down depending on your business. Amy: [00:27:14] Okay, great. So what advice would you have for a company or a business leader? Who's looking to potentially move their data operations into Kubernetes structure. Anil: [00:27:25] Yeah, that's a big question. Big talking to many of our customers. And from last few years initially they said they will, they want to adopt touch base on containers and Kubernetes for the use cases for test and development. Initially. And then that changed, like use cases, which are production use cases, but not mission critical, but technos more use cases. But then now I have seen that's the most are adopting this for a Q1 production use basis, mission critical use cases because of all the confidence they have bought through the platform. But mainly that it's a, what businesses all in our private practice. All right. Again, as I said earlier If you're already modernizing your platform, you have already a great you're already picked the containers and Kubernetes for your applications and your containerize, your applications. You're running your microservices, architecture on on Kubernetes. Then you want to. In that case, it makes total sense to run the database also in the same architecture, because then you can move that impotence mismatch. They can now manage both status and state applications together on the same platform. And in terms of business, it will be faster time to market. And ability to equity consolidation of multiple platforms. So it's like it helps in multiple areas. And also as a, we talked about, one of our goals is about being on the, have a hybrid cloud strategy or multi-cloud strategy. This Kubernetes will significantly help there. Amy: [00:28:53] Okay. So ask yourself, what is my business school essentially? Cool. All right. Great. All right. And now thank you very much for coming on the podcast. I appreciate your time. I thought this was great. Anil: [00:29:06] Yeah. Thank you so much. And like I said, thanks for having me in your show. Amy: [00:29:10] Oh, the one thing I want to ask you too, before we go, what, when did you release your book? Anil: [00:29:14] It was released back in in pretty 18 August. So we launched our product I want to do basically a mid-day splash to have a marketing splash. And so we launched this book it became the number three top book in 2019, but obviously it's been two years now, but yeah. And at one point it was a booklet already it as a hour as a number three book. Oh, Amy: [00:29:41] that's so exciting. Congratulations. Okay, great. What's the book called Anil: [00:29:45] it's. It's called coach based on Kubernetes. It is available on our site as Amy: [00:29:50] well as creative, very creative title. Okay, great. And now, where can we find you and what you're working on online? Yeah, Anil: [00:30:00] Definitely you can find me on LinkedIn. And after you on that, and then as well as on Twitter yeah if and you can definitely reach out to me at dot com. Amy: [00:30:10] Okay, perfect. Thank you very much. If you like this episode of the hacker noon podcast, don't forget to do all the things like it. Share it, subscribe to it, share it with your friends. This episode was edited by audio wizard, Alex hosted by me, Amy, Tom, and produced by hacker noon. Stay weird and I'll see you on the internet. Goodbye.