Introduction The open source community has built many implementations for deploying in Kubernetes. However, these implementations lack the crucial enterprise features such as — Backups, Automation, Alerting and, Monitoring. Starting with the MongoDB Ops Manager v4.0, MongoDB officially supports the management and deployment of MongoDB in Kubernetes with Backup, Automation, Alerting and, Monitoring. Thanks to ‘ ’ for integrating Kubernetes and MongoDB Ops Manager. MongoDB MongoDB Enterprise Operator (beta) for Kubernetes This is one of the many articles in multi-part series, , solely created for you to master by learning . In a few series of articles, I would like to give various tips to help you master MongoDB in Kubernetes. This article discusses , a new feature in MongoDB Ops Manager v4.0 — its applications, use case scenarios and finally some hands-on lab exercises. Mastering MongoDB — One tip a day MongoDB ‘one tip a day’ MongoDB Enterprise Operator for Kubernetes Mastering — MongoDB in Kubernetes What in MongoDB Enterprise Operator The is a MongoDB application-specific controller that extends the Kubernetes API to create, configure, and manage instances of MongoDB deployments on behalf of a Kubernetes user. The operator lets you deploy and manage MongoDB by accessing the Kubernetes API and MongoDB Ops Manager API. It automates common tasks by triggering a series of actions to achieve the desired state. For example, when you create MongoDB Kubernetes resource , then the operator MongoDB Enterprise Operator for Kubernetes MongoDbReplicaSet Automates the provisioning of StatefulSets for the MongoDB replica set Creates a deployment in MongoDB Ops Manager project Configures Ops Manager for Backup, Automation, Alerting and Monitoring (BAAM!) And many more features to come in the future. MongoDB enterprise operator architecture diagram The MongoDB Enterprise Operator for Kubernetes architecture diagram The above diagram shows various components involved in the process and illustrates how they interact with each other. The runs in a Kubernetes cluster and intercepts the requests to create resources. When a request comes in, it triggers a series of actions as shown below MongoDB enterprise operator MongoDB Reads the ConfigMap containing the Ops Manager baseUri and projectId Reads the Secret containing the Ops Manager publicApiKey Provisions the Pods required for MongoDB replica set using a MongoDB container image Each MongoDB container image runs an Ops Manager Automation agent Ops Manager Automation agent reaches out Ops Manager for it’s configuration state Ops Manager gives out desired state for each agent Ops Manager Automation agent works towards reaching the goal state. The agent downloads / installs mongod, Monitoring and Backup agents Hands-On lab exercises This lab exercise helps you understand how to make use of to deploy a MongoDB replica set using . MongoDB Enterprise Operator for Kubernetes kubectl Setup your local environment I have created some helper scripts to deploy the MongoDB in Kubernetes. First, let’s get started by downloading the source from my GitHub Setup your local environment repo I have created some helper scripts to deploy the MongoDB in Kubernetes. First, let’s get started by downloading the source from my GitHub repo wget -O k8-mongo.zip https://github.com/sarjarapu/k8-mongo/archive/master.zip unzip k8-mongo.zip k8-mongo-master # Download the helper scripts from GitHub repo cd A helper scripts to deploy the MongoDB in Kubernetes You would also need a Kubernetes and below set of tools installed in your environment. If you already have them installed, you may skip the next step. Kubernetes (GKE or minikube) kubectl kubernetes-helm If you are getting started with Kubernetes, then the will install all the required dependencies on a Mac OS. However, If you have any other operating system, please manually install all the dependencies for your OS. install.sh sh install.sh # Install and configure the dependencies for Mac OS # Dependencies: virtualbox, minikube, kubernetes-helm, bash-completion Install the helm chart for MongoDB enterprise operator The below script will install for Kubernetes using , the Kubernetes package manager. MongoDB Enterprise Operator helm wget -O master.zip https://github.com/mongodb/mongodb-enterprise-kubernetes/archive/master.zip unzip master.zip helm init --upgrade helm install mongodb-enterprise-kubernetes-master/helm_chart/ --name mongodb-enterprise # Download the helm chart from mongodb-enterprise-kubernetes GitHub repo # Initialize the helm and helm chart If is not available in your environment, you may install the operator using . helm kubectl kubectl apply -f mongodb-enterprise-kubernetes-master/mongodb-enterprise.yaml # If helm is not available # Install the enterprise operator via YAML Ops Manager configuration The is supported starting from MongoDB Ops Manager v4.0+. So, please ensure that you have installed in your environment. MongoDB Enterprise Operator Ops Manager v4.0 Note: The instructions for installing the Ops Manager v4.0 is out of scope for this article. You may refer to the to install the Ops Manager or use the MongoDB as an alternate solution. documentation Cloud Manager Once the Ops Manager is up and running, please follow the below instructions and have the information handy for the next step Make a note of the Ops Manager base URI Create a Project in Ops Manager Make a note of the Project ID Settings > Project Settings > General > Project ID Create a Public API key Account > Public API Access > API Keys Find your public IP address Whitelist your IP Address Account > Public API Access > API Whitelist Configure Version Manager with MongoDB versions of your interest Deployments > More > Version Manager Make note of the Ops Manager Project ID and Create Public API Key & Whitelist your IP address Update the scripts for your environment Please edit the file with appropriate values for your environment. You must update the values for the properties before you start creating a replica set. The required set of properties are as shown below templates/environment.sh OM_PROJECT_ID= OM_USER_BASE64=$( | base64) OM_API_KEY_BASE64=$( | base64) OM_URL= K8_NAMESPACE= MONGODB_VERSION= "<opsmanager_project_id>" echo "<opsmanager_userid>" echo "<opsmanager_public_apikey>" "<opsmanager_uri>" "<kubernetes_namespace>" "<mongodb_version>" For example, If you want to use MongoDB Cloud Manager and create a MonogDB v3.6.5 replica set then you must update the properies in the file as shown below. templates/environment.sh OM_URL= K8_NAMESPACE= MONGODB_VERSION= "https://cloud.mongodb.com/" "mongodb-world" "3.6.5" Create a MongoDB replica set in Kubernetes I have provided a sample template file for your convenience. Using the values defined in the environment file, and the template file, , you could easily generate a new custom YAML file, . You would need to run the above YAML file via to create the MongoDB replica set. templates/environment.sh templates/generate-yaml-simple-replicaset.sh samples/<kubernetes_namespace>-replicaset.yaml kubectl sh templates/generate-yaml-simple-replicaset.sh templates/environment.sh kubectl apply -f samples/ -replicaset.yaml # create the YAML file using the template # create the replica set based on the generated YAML source ${K8_NAMESPACE} Based on your download speed, the required resources/images are downloaded/created for you within a few seconds to a few minutes. When you run get all resources you would be see that there are 3 Pods, 1 Statefulset with Desired state of 3 and 2 services were created for you in the namespace you provided. kubectl -n get all # display all the resources in the namespace $K8_NAMESPACE # NAME READY STATUS RESTARTS AGE # pod/mongodb-world-replicaset-0 1/1 Running 0 4m # pod/mongodb-world-replicaset-1 1/1 Running 0 4m # pod/mongodb-world-replicaset-2 1/1 Running 0 4m # # NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) # AGE # service/mongodb-world-replicaset-svc ClusterIP None <none> 27017/TCP # 4m # service/mongodb-world-replicaset-svc-external NodePort 10.43.252.153 <none> # 27017:31750/TCP 4m # # NAME DESIRED CURRENT AGE # statefulset.apps/mongodb-world-replicaset 3 3 4m Finally, if you check the Ops Manager projects, then you would see a MongoDB deployment with the above Pods is created and the Automation and Monitoring is turned on as well. Under the hood details of the helper script Let’s try to understand what happens under the hood when you run the generated YAML. The shell script creates a YAML file based on the environment variables you defined in the file. In my case, it generated a file called . It simplifies the overall process by performing below tasks generate-yaml-simple-replicaset.sh templates/environment.sh mongodb-world-replicaset.yaml Create a namespace in Kubernetes Create ConfigMap with the Ops Manager Project ID, baseUrl Create Secret with the base64 encoded values of Ops Manager Username and Public Api Key Create a MongoDbReplicaSet with 3 members, version: 3.6.5 # mongodb-world-replicaset.yaml contents --- apiVersion: v1 kind: Namespace metadata: name: "mongodb-world" --- apiVersion: v1 kind: ConfigMap metadata: name: "mongodb-world-project" namespace: "mongodb-world" data: projectId: "59*************6fed" baseUrl: "https://cloud.mongodb.com/" --- apiVersion: v1 kind: Secret metadata: name: "mongodb-world-credentials" namespace: "mongodb-world" type: Opaque data: user: "c2************UK" publicApiKey: "Nj******************zRiCg==" --- apiVersion: mongodb.com/v1 kind: MongoDbReplicaSet metadata: name: "mongodb-world-replicaset" namespace: "mongodb-world" spec: members: 3 version: "3.6.5" project: "mongodb-world-project" credentials: "mongodb-world-credentials" persistent: false # For testing only podSpec: cpu: '0.25' memory: 512 M storage: 2 G Connecting to MongoDB deployment The connectivity to the MongoDB deployment within the Kubernetes cluster is very simple. Here in this example, I am connecting to the interactive terminal of first replica set member and using the mongo shell to connect to localhost. kubectl -n -it -replicaset-0 -- bin/bash /var/lib/mongodb-mms-automation/mongodb-linux-x86_64-3.6.5/bin/mongo ${K8_NAMESPACE} exec ${K8_NAMESPACE} # mongodb@mongodb-world-replicaset-0:/$ # MongoDB shell version v3.6.5 # connecting to: mongodb://mongodb-world-replicaset-0:27017/ # MongoDB server version: 3.6.5 # mongodb-world-replicaset:PRIMARY> Since I am running these pods on the , the Node IP address is resolvable on my laptop. Therefore, I could connect to the MongoDB from outside of the cluster by using the Node Ports. However, you would need to use the Kubernetes ingress to allow connectivity from servers external to Kubernetes. minikube 10.128.0.7 kubectl -n describe pod/mongodb-world-replicaset-0 | grep kubectl -n get services mongo --host 10.128.0.7 --port 31750 ${K8_NAMESPACE} 'Node:' # Node: 10.128.0.7 ${K8_NAMESPACE} # NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE # mongodb-world-replicaset-svc ClusterIP None <none> 27017/TCP 41m # mongodb-world-replicaset-svc-external NodePort 10.43.252.153 <none> 27017:31750/TCP 41m # MongoDB shell version v3.6.5 # connecting to: mongodb://mongodb-world-replicaset-0:27017/ # MongoDB server version: 3.6.5 # mongodb-world-replicaset:PRIMARY> Remove the MongoDB Cluster The below command will delete the . The Ops Manager deployment in the given project should be removed as well. If for any reason this project still exists, you have to go to Deployment > … > to remove it completely. MongoDbReplicaSet Remove from Ops Manager kubectl delete -f samples/ -replicaset.yaml ${K8_NAMESPACE} Check Enterprise Operator logs for errors Sometimes you may notice that creation did not succeed. To figure out what went wrong, you would have to check the logs on the mongodb-enterprise-operator pod. MongoDbReplicaSet K8_OPERATOR_POD_NAME=$(kubectl -n mongodb get pods --selector=app=mongodb-enterprise-operator --output=jsonpath= ) kubectl -n mongodb logs # find the pod name for mongodb-enterprise-operator using selectors '{.items[0].metadata.name}' # display the mongodb-enterprise-operator logs from mongodb namespace $K8_OPERATOR_POD_NAME Some of the typical errors, you may incur are related to Current IP Address is not added to Whitelist MongoDB Version not checked/binaries not available on Ops Manager If you find such errors in your logs, then have to fix them first and recreate the operator pod before creating the replica set again. Since the operator is a deployment, you could simply delete the pod and a new one would be created for you. kubectl -n mongodb delete pod sleep 5 kubectl -n get all # delete the existing pod after fixing the issue $K8_OPERATOR_POD_NAME # display all the resources in the namespace ${K8_NAMESPACE} Key Points & Other challenges I am very positive that you are very excited if you got it working this far. Before you go ahead and deploy your first MongoDB deployment in your production Kubernetes, I want you to understand below key points. Pods are ephemeral. So make sure you use the persistent disks Make sure the Pods are provisioned across different nodes to ensure the High Availability Explicitly set the wiredTigerCacheSizeGB to 50% of Pod’s memory Use Affinity, Anti-Affinity to ensure Pods are distributed and resources are shared evenly Summary The support for deploying is just the beginning. The future release may add lot more features replica sets and clusters Scale up/down the number of members in the replica set Convert a replica set to sharded cluster Inject advanced mongod configurations, etc. The MongoDB Enterprise Operator for Kubernetes brings in a lot of excitement for the developers in you. However, I want to stress an important point. MongoDB offers redundancy and high availability of the database via replica sets. While the operator simplifies the deployment of MongoDB replica sets, you need to take proper measures while provisioning the Pods so that MongoDB is highly available across multiple Kubernetes Nodes and/or Data Centers. Most importantly the Pods should be allocated with the required hardware resources as per the sizing exercise. Now that you understand all the crucial points for deploying MongoDB in Kubernetes, you might wonder, “How to size my deployments (find out how much hardware resources are needed)?”. Great question! But that’s a topic for another day. Hopefully, you learned something new today on you scale the path to “ ”. Mastering MongoDB — One tip a day Resources — by Robert Walters Introducing the MongoDB Enterprise Operator for Kubernetes and OpenShift GitHub repo: mongodb-enterprise-kubernetes by Janakiram Kubernetes Webinars
