Custom Plugin Development For APISIX With Lua And ChatGPT

One of the key features of Apache APISIX is its extensibility through plugins. APISIX allows you to build your own custom plugin to add extra functionalities and manage the API traffic more efficiently. Oftentimes, you use Lua programming language to implement new plugins or leverage plugin runners to develop plugins in your favorite programming language. However, APISIX has the best support for Lua. After writing a couple of plugins for APISIX in Lua, I understood that you do not need to know the fundamentals of Lua programming or to be an expert in this language at all when your ChatGPT friend is always with you. For example, with my background in Java and C#, I can understand the code and logic written in Lua and I believe that you can do the same.

This article will guide you through the process of developing a new custom plugin called file-proxy for APISIX using Lua and ChatGPT (We use it to write some Lua code for us). This plugin will be used to expose the static files through API and fetch a file from a specified URL.

APISIX was built to enhance the existing functionalities of Nginx. And Nginx provides a bunch of reusable Lua modules that APISIX makes use of them.

New file proxy plugin use-case

Before jumping into the actual plugin implementation, let’s understand first why we need this plugin. At the time of writing this post, APISIX might not provide a built-in plugin for a similar case. That’s why we are going to build a new one. Often, we want to expose a static file (Yaml, JSON, JavaScript, CSS, or image files) through API.

For example, APISIX API Gateway acts as a front door in your application to route incoming requests to multiple API endpoints, it is the right place to define all server URLs, paths, parameters, descriptions of each API endpoint and their inputs and outputs. And you build OpenAPI specifications to document the API. OpenAPI .yaml file is like a map that guides your API user in understanding and interacting with your API. By providing the path of openapi.yaml file (where it is stored in your server) to the plugin, you can fetch and serve the file directly through your API gateway, providing a consistent interface for API consumers. Then your API users can access .yaml file at the specified URL (https://example.com/openapi.yaml).

There are other use cases as well, you might think of using this file-proxy plugin for a simple Content Delivery Network (CDN) replacement. If you have a small-scale application and don't want to use a full-fledged CDN, you can use the file-proxy plugin to serve static files from a specific location. The file-proxy plugin can be used as a caching layer for files. If you have files that are expensive to fetch or generate, you can use the plugin to fetch the file once and then serve the cached version for subsequent requests.

Steps to develop the file-proxy plugin

We are going to run APISIX locally and our API Gateway will be hosted on http://localhost:9080. When the development is ready, you can deploy it to your server or any cloud provider. Basically, we want to place a file openapi.yaml to http://localhost:9080/openapi.yaml path. You will learn how to achieve this.

Prerequisites

Before you start, it is good to have a basic understanding of APISIX. Familiarity with API gateway, and its key concepts such as routes, upstream, Admin API, plugins, and HTTP protocol will also be beneficial.
Docker is used to installing the containerized etcd and APISIX.
curl is used to send requests to APISIX Admin API. You can also use tools such as Postman to interact with the API.

Understand the demo project and files

We will leverage the existing file-proxy demo project on GitHub. It has a quite similar structure to the existing Apisix docker example repo, only we removed unnecessary files to keep the demo simple. The project has 3 folders, docker-compose.yml, and sample openapi.yaml files.

docker-compose.yml defines two containers one for APISIX and another for etcd (which is configuration storage for APISIX).
custom-plugins folder has the implementation of the file-proxy plugin in Lua. We review it in the following sections.
openapi.yaml is just a sample OpenAPI specification we expose.

Implement file-proxy plugin

We start by asking ChatGPT, it gives us a guide almost close to the real implementation but the answer is too abstract and when you follow the process, you will end up with a non-working plugin. However, it helps us to extract useful Lua code. If we know the real process of developing plugins, it will be easier to combine both knowledge in practice.

Implement file-proxy plugin with ChatGPT

1. Create a Lua File

We create a new empty Lua file in the /custom-plugins directory of the project. The name of the file should be the name of our plugin. For example, if your plugin is named file-proxy, you should create a file named file-proxy.lua.

2. Register the plugin in APISIX

APISIX needs to know where this plugin file is located and is able to run the plugin accordingly. To do so, we should first define the file path where APISIX finds file-proxy.lua file by adding the file path to the extra_lua_path attribute of APISIX in the config.yaml.

apisix:
  extra_lua_path: "/opt/?.lua"
  node_listen: 9080

Now you may ask why the file path is set to /opt/?.lua. Because we run APISIX using docker. You may notice this in the docker-compose.yml file there are 3 volumes ./custom-plugins:/opt/apisix/plugins:ro

volumes:
      - ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro
      - ./openapi.yaml:/usr/local/apisix/conf/openapi.yaml:ro
      - ./custom-plugins:/opt/apisix/plugins:ro

This mounts the local directory./custom-plugins where our file-proxy.lua file with the custom plugin implementation as a read-only volume in the docker container at the path /opt/apisix/plugins. This allows the custom plugin to be added to APISIX in the runtime to another path in Docker which is inside /opt/?.lua. Similarly, the other two files we copied to Docker folders.

Next step, we enable the plugin in the APISIX plugins list. This is done by adding the plugin name to the plugins list in the APISIX configuration file (config.yaml):

plugins:       
  - file-proxy

Note that this action will override all existing default plugins specified in config-default.yaml. You need to add manually other plugins by their name if you want to use your custom plugin with the combination.

3. File proxy plugin Lua code breakdown

Up to now, we only registered the plugin that simply does nothing. It is time to implement it. The plugin logic is implemented as Lua functions. You can check how it is done in file-proxy.lua.

Let’s break down the file-proxy.lua file to better understand the structure of the code and flow that helps you to create new plugins on your own. You can simply ask ChatGPT to explain the Lua code:

explain the Lua code with ChatGPT

Actually, we got quite a good explanation of the code (Because it was partially written by ChatGPT).

Image description

I will only walk you through the important parts of this code so that you are not lost or fully rely on AI to write your plugins.

4. Plugin file structure

Every plugin Lua file should have the following structure:

1. Modules: You import the necessary modules/libraries we need for the plugin

local core     = require("apisix.core")
...

2. Plugin name: Every plugin has a unique name, it can be the same as our Lua file name.

local plugin_name = "file-proxy"

3. Plugin schema: Every plugin has a plugin schema, where we usually specify inputs to the plugin. The input we will pass from the APISIX route configuration, which you can see later when we test the plugin. For the file-proxy plugin, the plugin needs a file path to read the file and return a response so our parameter is the path which string type. You understand the schema like a method declaration with params in other programming languages.

local plugin_schema = {
    type = "object",
    properties = {
        path = {
            type = "string" -- The path of the file to be served
        },
    },
    required = {"path"} -- The path is a required field
}

4. Plugin definition: It is a really important part of plugin implementation that we define as a table with properties for the version, priority, name, and schema. The name and schema are the plugin's name and schema defined earlier. The version and priority are used by APISIX to manage the plugin. The version typically refers to the version that is currently in use like API versioning. If you publish and update your plugin logic, it is going to be 1.1 (You can set any version you wish). But you need to be very careful in choosing priority. The priority field defines in which order and phase your plugin should be executed. For example, the 'ip-restriction' plugin, with a priority of 3000, will be executed before the 'example-plugin', which has a priority of 0. This is due to the higher priority value of the 'ip-restriction' plugin. If you're developing your own plugin, make sure that you followed the order of plugins not to mess up the order of existing plugins. You can check the order of existing plugins in the config-default.yaml file and open the Apache APISIX Plugin Development Guide to determine.

local _M = {
    version = 1.0,
    priority = 1000,
    name = plugin_name,
    schema = plugin_schema
}

5. Schema check: The check_schema Lua function is used to validate the plugin in a route configuration (You will see it soon in the test section) against the plugin schema we defined earlier.

-- Function to check if the plugin configuration is correct
function _M.check_schema(conf)
  -- Validate the configuration against the schema
  local ok, err = core.schema.check(plugin_schema, conf)
  -- If validation fails, return false, and the error
  if not ok then
      return false, err
  end
  -- If validation succeeds, return true
  return true
end

6. Plugin logic: access function is the core function where we can write the major plugin logic. It is called during the access phase of the Nginx request processing pipeline and we control the traffic and write custom instructions. For file-proxy, we need to open the file specified in the plugin configuration, read its content, and return the content as the response. If the file cannot be opened, it logs an error and returns a 404 Not Found status. It is the exact place we give this work to ChatGPT:

Image description

After we structured and refactored the code, below is how it looks like:

function _M.access(conf, ctx)
  -- Open the file specified in the configuration
  local fd = io.open(conf.path, "rb")
  -- If the file is opened successfully, read its content and return it as the response
  if fd then
    local content = fd:read("*all")
    fd:close()
    ngx.header.content_length = #content
    ngx.say(content)
    ngx.exit(ngx.OK)
  else
    -- If the file cannot be opened, log an error and return a 404 Not Found status
    ngx.exit(ngx.HTTP_NOT_FOUND)
    core.log.error("File is not found: ", conf.path, ", error info: ", err)
  end
end

7. Logging logic: It is always preferable to log plugin configuration so that we can debug and check if the plugin is working as we expected. We can log requests to the plugin and responses.

-- Function to be called during the log phase
function _M.log(conf, ctx)
    -- Log the plugin configuration and the request context
    core.log.warn("conf: ", core.json.encode(conf))
    core.log.warn("ctx: ", core.json.encode(ctx, true))
end

Install Apache APISIX

Once we learned how to develop our custom file-proxy plugin, registered in APISIX. Now it is time to test the plugin. You can easily install the apisix-file-proxy-plugin-demo project by running docker compose up from the project root folder after you fork/clone the project.

Create a route with the file-proxy plugin

To use and test our new file-proxy plugin we need to create a route in APISIX that uses the plugin:

curl "http://127.0.0.1:9180/apisix/admin/routes/open-api-definition" -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
   "name":"OpenAPI Definition",
   "desc":"Route for OpenAPI Definition file",
   "uri":"/openapi.yaml",
   "plugins":{
      "file-proxy":{
         "path":"/usr/local/apisix/conf/openapi.yaml"
      }
   }
}'

You can ask ChatGPT to explain the above configuration:

Image description

Test the plugin

Then, you can send a cURL request to the route or open the link 127.0.0.1:9080/openapi.yaml in your browser. The response should be the content of the file openapi.yamlat the specified URL.

curl -i http://127.0.0.1:9080/openapi.yaml

The plugin works as we expected. With this plugin configuration, you can now access any files using the specified route.

Summary

Developing custom plugins for APISIX with Lua is a powerful way to extend the functionality of the API gateway. We demonstrated how to create the file-proxy plugin in this post, defined the plugin definition and schema, validated the plugin configuration, and implemented custom logic during the access and log phases of the request processing pipeline in APISIX. ChatGPT helped us to write Lua code for the main functionality by filling our lacking knowledge of this programming language. Happy coding!