is an API Gateway, which builds upon the reverse-proxy to offer a plugin-based architecture. The main benefit of such an architecture is that it brings structure to the configuration of routes. It’s a help at scale, when managing hundreds or thousands of routes. Apache APISIX OpenResty In this post, I’d like to describe how plugins, priority, and phases play together and what pitfalls you must be aware of. APISIX plugin’s priority When you configure a route with multiple plugins, Apache APISIX needs to execute them in a order so that the results are the same over time. For this reason, every APISIX plugin has a . You can check a plugin priority directly in the code. For example, here’s the relevant code fragment for the plugin: consistent hardcoded priority basic-auth local _M = {
    version = 0.1,
    priority = 2520,                                 #1
    type = 'auth',
    name = plugin_name,
    schema = schema,
    consumer_schema = consumer_schema
} Priority is 2520 For documentation purposes, the file lists the priority of all out-of-the-box plugins. Here’s an excerpt: default-config.yaml plugins:
  - ldap-auth                      # priority: 2540
  - hmac-auth                      # priority: 2530
  - basic-auth                     # priority: 2520
  - jwt-auth                       # priority: 2510
  - key-auth                       # priority: 2500
  - consumer-restriction           # priority: 2400
  - forward-auth                   # priority: 2002
  - opa                            # priority: 2001
  - authz-keycloak                 # priority: 2000
  #- error-log-logger              # priority: 1091
  - proxy-cache                    # priority: 1085
  - body-transformer               # priority: 1080
  - proxy-mirror                   # priority: 1010
  - proxy-rewrite                  # priority: 1008
  - workflow                        # priority: 1006
  - api-breaker                    # priority: 1005
  - limit-conn                     # priority: 1003
  - limit-count                    # priority: 1002
  - limit-req                      # priority: 1001 Imagine a route configured with and . Because of their respective priority, would run before . proxy-mirror proxy-rewrite proxy-mirror proxy-rewrite upstream:
  - id: 1
    nodes:
      "oldapi:8081": 1

routes:
  - id: 1
    uri: "/v1/hello*"
    upstream_id: 1
    plugins:
      proxy-rewrite:                                 #1-3
        regex_uri: ["/v1(.*)", "$1"]
      proxy-mirror:                                  #2
        host: http://new.api:8082 The plugin ordering in this file is not relevant has priority 1010 and will execute first proxy-mirror has priority 1008 and will run second proxy-rewrite The above setup has an issue. For example, if we call , APISIX will first mirror the request, and then remove the prefix. Hence, the new API receives instead of . It’s possible to override the default priority to fix it: localhost:9080/v1/hello /v1 /v1/hello /hello routes:
  - id: 1
    uri: "/v1/hello*"
    upstream_id: 1
    plugins:
      proxy-rewrite:
        _meta:
          priority: 1020                             #1
        regex_uri: ["/v1(.*)", "$1"]
      proxy-mirror:
        host: http://new.api:8082 Override the default priority Now, has higher priority than : the former runs before the latter. proxy-rewrite proxy-mirror In this case, it works flawlessly; in others, it might not. Let’s dive further. APISIX phases APISIX runs plugins not only according to their priorities but also through dedicated phases. Because APISIX builds upon OpenResty, which builds upon ngxinx, the phases are very similar to the phases of these two components. nginx defines several an HTTP request goes through: phases NGX_HTTP_POST_READ_PHASE NGX_HTTP_SERVER_REWRITE_PHASE NGX_HTTP_FIND_CONFIG_PHASE NGX_HTTP_REWRITE_PHASE NGX_HTTP_POST_REWRITE_PHASE NGX_HTTP_PREACCESS_PHASE NGX_HTTP_ACCESS_PHASE NGX_HTTP_POST_ACCESS_PHASE NGX_HTTP_PRECONTENT_PHASE NGX_HTTP_CONTENT_PHASE NGX_HTTP_LOG_PHASE Each phase focuses on a task, , verifies that the client is authorized to make the request. i.e. NGX_HTTP_ACCESS_PHASE In turn, OpenResty offers similarly named phases. From nginx documentation: Finally, here are the phases of Apache APISIX: rewrite access before_proxy header_filter body_filter log We have seen two ways to order plugins: and Now comes the most important rule: ! by priority by phase. order by priority only works inside a phase For example, the plugin has a priority of 900 and runs in phase ; the plugin has a priority of 995 and runs in phase . Regardless of their respective priorities, will happen before , because happens before . Likewise, changing a priority won’t "move" a plugin out of its phase. redirect rewrite gzip body_filter redirect gzip rewrite body_filter The example above with and worked because both run in the phase. proxy-mirror proxy-rewrite rewrite The main issue is that priority is documented in the file, while the phase is buried in the code. Worse, some plugins run across different phases. For example, let’s check the proxy plugin and, more precisely, the functions defined there: config-default.yaml proxy-rewrite local function is_new_headers_conf(headers) local function check_set_headers(headers) function _M.check_schema(conf) function _M.rewrite(conf, ctx) The name of the function is suspiciously similar to one of the phases above. Looking at other plugins, we see the same pattern repeat. Apache APISIX runs plugin functions with the same name as the phase. rewrite() I took the liberty of summarizing all plugins and their respective phases in a table. +----------------------+---------+--------+-------------+--------------+------------+-----+
| Plugin               | Phase                                                            |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| General              | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| redirect             | X       |        |             |              |            |     |
| echo                 |         |        |             | X            | X          |     |
| gzip                 |         |        |             | X            |            |     |
| real-ip              | X       |        |             |              |            |     |
| ext-plugin-pre-req   | X       |        |             |              |            |     |
| ext-plugin-post-req  |         | X      |             |              |            |     |
| ext-plugin-post-resp |         |        | X           |              |            |     |
| workflow             |         | X      |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Transformation       | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| response-rewrite     |         |        |             | X            | X          |     |
| proxy-rewrite        | X       |        |             |              |            |     |
| grpc-transcode       |         | X      |             | X            | X          |     |
| grpc-web             |         | X      |             | X            | X          |     |
| fault-injection      | X       |        |             |              |            |     |
| mocking              |         | X      |             |              |            |     |
| degraphql            |         | X      |             |              |            |     |
| body-transformer     | X       |        |             | X            | X          |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Authentication       | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| key-auth             | X       |        |             |              |            |     |
| jwt-auth             | X       |        |             |              |            |     |
| basic-auth           | X       |        |             |              |            |     |
| authz-keycloak       |         | X      |             |              |            |     |
| authz-casdoor        |         | X      |             |              |            |     |
| wolf-rbac            | X       |        |             |              |            |     |
| openid-connect       | X       |        |             |              |            |     |
| cas-auth             |         | X      |             |              |            |     |
| hmac-auth            | X       |        |             |              |            |     |
| authz-casbin         | X       |        |             |              |            |     |
| ldap-auth            | X       |        |             |              |            |     |
| opa                  |         | X      |             |              |            |     |
| forward-auth         |         | X      |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Security             | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| cors                 | X       |        |             | X            |            |     |
| uri-blocker          | X       |        |             |              |            |     |
| ua-restriction       |         | X      |             |              |            |     |
| referer-restriction  |         | X      |             |              |            |     |
| consumer-restriction |         | X      |             |              |            |     |
| csrf                 |         | X      |             | X            |            |     |
| public-api           |         | X      |             |              |            |     |
| chaitin-waf          |         | X      |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Traffic              | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| limit-req            |         | X      |             |              |            |     |
| limit-conn           |         | X      |             |              |            | X   |
| limit-count          |         | X      |             |              |            |     |
| proxy-cache (init)   |         | X      |             | X            | X          |     |
| proxy-cache (disk)   |         | X      |             | X            |            |     |
| proxy-cache (memory) |         | X      |             | X            | X          |     |
| request-validation   | X       |        |             |              |            |     |
| proxy-mirror         | X       |        |             |              |            |     |
| api-breaker          |         | X      |             |              |            | X   |
| traffic-split        |         | X      |             |              |            |     |
| request-id           | X       |        |             | X            |            |     |
| proxy-control        | X       |        |             |              |            |     |
| client-control       | X       |        |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Observability        | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| zipkin               | X       | X      |             | X            |            | X   |
| skywalking           | X       |        |             |              |            | X   |
| opentelemetry        | X       |        |             |              |            | X   |
| prometheus           |         |        |             |              |            |     |
| node-status          |         |        |             |              |            |     |
| datadog              |         |        |             |              |            | X   |
| http-logger          |         |        |             |              | X          | X   |
| skywalking-logger    |         |        |             |              |            | X   |
| tcp-logger           |         |        |             |              |            | X   |
| kafka-logger         |         |        |             |              | X          | X   |
| rocketmq-logger      |         |        |             |              | X          | X   |
| udp-logger           |         |        |             |              |            | X   |
| clickhouse-logger    |         |        |             |              | X          | X   |
| syslog               |         |        |             |              |            | X   |
| log-rotate           |         |        |             |              |            |     |
| error-log-logger     |         |        |             |              |            |     |
| sls-logger           |         |        |             |              |            | X   |
| google-cloud-logging |         |        |             |              |            | X   |
| splunk-hec-logging   |         |        |             |              |            | X   |
| file-logger          |         |        |             |              | X          | X   |
| loggly               |         |        |             |              | X          | X   |
| elasticsearch-logger |         |        |             |              |            | X   |
| tencent-cloud-cls    |         | X      |             |              | X          | X   |
| loki-logger          |         |        |             |              | X          | X   |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Others               | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| serverless           |         | X      |             |              |            |     |
| openwhisk            |         | X      |             |              |            |     |
| dubbo-proxy          |         | X      |             |              |            |     |
| kafka-proxy          |         | X      |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+ Conclusion I’ve detailed Apache APISIX plugin phases and priorities in this post. I’ve explained their relationship with one another. Icing on the cake, I documented each out-of-the-box plugin’s phase(s). I hope it will prove helpful. To go further: Default plugin priorities Apache APISIX plugin execution lifecycle Plugin execution code Originally published at , on December 10th, 2023 A Java Geek

Augmentastic || Augmented Reality

Five Apache Projects You Don't (But Should) Know About

Implementing Canary Releases with Apache APISIX

Read all my posts!

Navigating Apache APISIX: A Strategic Approach to Plugin Priorities and Execution Phases

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

16 Best Practices For Securing Your APIs with Apache APISIX - Part 1

5 Web3 Trends That I'm Keeping an Eye on for the Rest of 2023

Abstraction Layers and API's for Cloud Native Environments

Associative Links Explained

Code simplicity: A language independent perspective

16 Best Practices For Securing Your APIs with Apache APISIX - Part 1

5 Web3 Trends That I'm Keeping an Eye on for the Rest of 2023

Abstraction Layers and API's for Cloud Native Environments

Associative Links Explained

Code simplicity: A language independent perspective

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps