is an excellent key-value distributed database used internally by Kubernetes and managed by the CNCF. It's a great option, and that's the reason why Apache APISIX uses it too. Yet, it's not devoid of issues. etcd First, some mention scalability, but one can expect this from a distributed data store that values consistency. Another issue may be the need for more familiarity with etcd. It's relatively new, so your Ops team may need help operating it correctly while having decades of operating MySQL or Postgres. Finally, only a few etcd users are aware that it lacks maintainers: In the last few months, primary maintainers G.L. (Amazon, announcement) and S.B. (Red Hat) have stopped actively participating in the project. This leaves the project with only one active and two occasionally-reviewing maintainers, M.S. (Google),  P.T. (Google), both are relatively new to the project (1 month and 1 year of tenure) and S.P.Z. (IBM). Other maintainers are either dormant or have very minimal activity over the last six months. . The project is effectively unmaintained (emphasis mine) -- Google Groups of Kubernetes Steering Committee, March 2022 For all those reasons, you may prefer to use a standard SQL database with Apache APISIX. In this post, I'll show how you can use MySQL. The kine project It would be a lot of effort if each product had to introduce an abstraction layer and different adapters for both etcd and other databases. kine is a project that aims to offer a translation step between etcd calls and other implementations: Kine is an etcdshim that translates etcd API to: SQLite Postgres MySQL NATS Jetstream Features Can be ran standalone so any k8s (not just K3s) can use Kine Implements a subset of etcdAPI (not usable at all for general purpose etcd) Translates etcdTX calls into the desired API (Create, Update, Delete) -- Kine (Kine is not etcd) In essence, kine is a Go library that translates etcd calls to the datastore you want (among those implemented). Yet, using kine directly is a non-trivial effort. Fortunately, , the company that gave Apache APISIX to the Apache Software Foundation, provides a component already focused on APISIX usage. api7 ETCD adapter ETCD adapter wraps kine to be APISIX-specific: ETCD Adapter mimics the ETCD V3 APIs best effort. It incorporates the kine as the Server side implementation, and it develops a totally in-memory watchable backend. Not all features in ETCD V3 APIs supported, this is designed for Apache APISIX, so it's inherently not a generic solution. -- ETCD adapter Two things of note: At the moment of this writing, the adapter supports either local in-memory storage or MySQL It's available as an embeddable library but also as a standalone component Therefore, we can design our architecture as the following: Demo Let's implement the above architecture with an additional admin UI over MySQL. I'll use Docker Compose: version: "3"

services:
  apisix:
    image: apache/apisix:3.4.0-debian                         #1
    volumes:
      - ./config.yaml:/usr/local/apisix/conf/config.yaml:ro
    ports:
      - "9080:9080"
      - "9180:9180"
    depends_on:
      - etcd-adapter
    restart: always                                           #2
  etcd-adapter:
    build: ./etcd-adapter                                     #3
    volumes:
      - ./adapter.yml:/etcd-adapter/conf/config.yaml:ro       #4
    depends_on:
      - mysql
    restart: always                                           #2
  mysql:
    image: bitnami/mysql:8.0                                  #5
    ports:
      - "3306:3306"
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_USER: etcd
      MYSQL_PASSWORD: etcd
      MYSQL_DATABASE: apisix
  adminer:
    image: adminer:standalone                                 #6
    ports:
      - "8080:8080"
    environment:
      ADMINER_DEFAULT_SERVER: mysql
    depends_on:
      - mysql Latest version of Apache APISIX, yeah! To avoid any failure with dependencies between containers, restart until it works. Kubernetes's manifests would involve health checks still needs to provide a container. We need to build from the source code api7.ai Override the default configuration file with a context-specific one The regular MySQL image didn't work for me. Let's take the one from Bitnami Adminer, formerly known as PHP myAdmin, will help to visualize the database state ETCD-adapter's configuration looks like this: server:
  host: 0.0.0.0                 #1
  port: 12379

log:
  level: info

datasource:
  type: mysql                   #2
  mysql:
    host: mysql                 #3
    port: 3306                  #3
    username: etcd              #3
    password: etcd              #3
    database: apisix Bind any IP since Docker will assign a random one Implementation type. The default is ; we need to change it. btree As configured in the file docker-compose.yml Finally, here's Apache APISIX configuration: deployment:
  admin:
    allow_admin:
      - 0.0.0.0/0
  etcd:
    host:
      - "http://etcd-adapter:12379"   #1 Use this etcd instance, which is the adapter Testing Now that we are set let's test our system by creating a route: curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '{                                          
  "methods": ["GET"],
  "uris": ["/get"],
  "upstream": {
    "nodes": {
      "httpbin.org:80": 1
    }
  }
}' http://localhost:9180/apisix/admin/routes/1 We can now get it: curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' http://localhost:9180/apisix/admin/routes/1 We can also check via the Adminer interface that it has been persisted via MySQL: Unfortunately, we need to stop at this point. Getting all routes doesn't work: curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' http://localhost:9180/apisix/admin/routes {"header":{"revision":"1689689596"},"message":"Key not found"} Worse, using the route fails: curl localhost:9080/get {"error_msg":"404 Route Not Found"} Conclusion etcd is an excellent piece of infrastructure Kubernetes uses, but there might be better choices in some contexts. Worse, it might become a security threat in the future - or is already one, because of the lack of maintenance. Being able to move away from etcd is a considerable benefit. kine offers an etcd-compatible facade and multiple implementations. Using kine with Apache APISIX requires some adaptation effort, already done in ETCD-Adapter. Currently, ETCD-Adapter is not feature-complete (to say the least) and requires more love. That's why it was not donated to the Apache Foundation yet. If you're a Go developer and are interested in the project, feel free to subscribe to the to offer your help. Apache APISIX mailing list and/or join our Slack The complete source code for this post can be found on . GitHub To go further: Kine ETCD Adapter "Goodbye etcd, Hello PostgreSQL: Running Kubernetes with an SQL Database" Originally published at on July 23rd, 2023 A Java Geek

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Apache APISIX Without etcd?

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Object Creation: The Issues I Faced and the Patterns That Helped Me

Apache APISIX vs. Spring Cloud Gateway

Creating a Poor Man's API Using PostgREST

Creating a Simple gRPC Service on the Client Side

Demo: Sticky Sessions with Apache APISIX for Improved Load Balancing

Object Creation: The Issues I Faced and the Patterns That Helped Me

Apache APISIX vs. Spring Cloud Gateway

Creating a Poor Man's API Using PostgREST

Creating a Simple gRPC Service on the Client Side

Demo: Sticky Sessions with Apache APISIX for Improved Load Balancing

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps