Apache APISIX’s technology selection, testing and continuous integration

Share on facebook
Share on google
Share on twitter
Share on linkedin

Author:Ming Wen
Apache APISIX

Apache APISIX is a cloud-native microservices API gateway, delivering the ultimate performance, security, open source and scalable platform for all your APIs and microservices.

Apache APISIX is based on Nginx and etcd. Compared with traditional API gateways, Apache APISIX has dynamic routing and plug-in hot loading, which is especially suitable for API management under micro-service system.

This article mainly mainly includes three aspects:

  1. First, what is API gateway and what does it do?
  2. Second, how to develop API gateway?
  3. Third, Apache APISIX is an open source project. It introduces how we do tests and conduct continuous integration when there are only two people. It involves some common things of OpenResty, which I think is worth learning.

What is API gateway?

In short, the API gateway makes the common functions of multiple services as an intermediate layer, such as statistics, security, rate-limiting, speed-limiting, and caching. In this way, each service layer only focuses on its own business, with irrelevant needs thrown onto the API gateways.

Traditional functions of API Gateway

  1. Process API requests more securely and efficiently. Traditional API gateways have some basic functions that are applicable until now. API internet traffic,regardless of north-south or east-west, can be processed quickly and securely. This is the original purpose of API gateways.
  2. Covers all functions of Nginx. Nginx’s functions are all covered by API gateway, including reverse proxy, load balancing, caching, secure authentication, rate-limiting, speed-limiting, etc.
  3. Support functions that ordinary web servers cannot achieve, such as dynamic upstream, dynamic SSL certificate, dynamic rate-limiting and speed-limiting, active/passive health check, service disconnection, etc., which are not supported by traditional web servers such as Nginx and Apache.
  4. Provide lifecycle management. In the API gateway field, Google is the biggest player. Google now integrates all the functions on Google Cloud after acquiring Apigee in 2016. In addition to these functions like reverse proxy, load balancing, and rate-limiting and speed-limiting, API gateways also include the design, file management and testing of API. The series of functions from project design to product test are all included in API gateway’s lifecycle management.

New Features under Cloud Native Computing

Why Apache APISIX redo traditional products? This is because there exists some new changes in users and technology architecture under cloud native computing and microservice systems.

New Changes of Technology Architecture:

  1. Need to interface with important components such as Prometheus, Zipkin, Skywalking, etc.
  2. gRPC proxy and protocol transcoding (REST <=> gRPC): now HTTP is less used in microservices. People start to use gRPC, so the conversion from HTTP to gRPC and gRPC proxy are highly required.
  3. Change in identity authentication: In traditional Nginx, generally, after the traffic comes in, it performs the function of reverse proxy and load balancing according to route rules. The identity of the client that sends the traffic is rarely authenticated. However, in cloud native computing, because a lot of traffic is in microservices, strict identity authentication is a must, including encryption and OpenID, which can be finished by third-party companies.
  4. Serverless is also a very famous concept in recent years. For example, if you want to dynamically run a function or stop a function at an edge node, or dynamically change the content, you can deploy API gateways at edge nodes. With FaaS (Function as a Service), edge nodes will be more flexible. Apache APISIX has recently supported Serverless, which can let one of Lua functions run dynamically at the edge nodes.
  5. Keeping stateless, expanding and reducing capability randomly: API gateways didn’t have such high- performance requirements more than a decade ago. At that time Internet traffic mainly runs from the browser to the server, and there was less traffic from microservices or intranet. However, there is a lot of traffic now, because quantities of 4G/5G mobile phones and IoT devices access servers. Therefore, higher-performance API gateways are needed to support them. An important standard under cloud native computing is that all services can be expanded and reduced at will through containers, which is friendly to Kubernetes.
  6. Support multiple clouds and hybrid clouds: Now it is a trend to turn to cloud usage. Generally, we do not put services only on one cloud. We may put some services on Tencent Cloud, some on Google Cloud, some on Alibaba Cloud, and others on private clouds. According to their security and prices, we may make some adjustments. For example, private data which needs securer protection will be stored on private clouds, while some resources related to CDN will be placed on a cheaper cloud. In order to help us choose clouds efficiently, vendor-independent API gateways are required to make distributions.

Applications of API Gateways

What can new API Gateways do? I’ll give you two examples.

  1. Replace all Nginx functions to handle north-south traffic

It has obvious advantages over Nginx and is implemented completely dynamically. Whether it is Kong or Apache APISIX, being dynamic is the core. Modifying any configuration file in Nginx requires reloading to take effect.

Let’s imagine a scenario. Use Nginx as the foremost route to do the load balancing. When sudden traffic comes, a lot of upstream servers need to be added quickly. The Nginx configuration files must be constantly modified and reloaded. After the traffic has passed, we need to remove the upstream servers, remodify Nginx files and reload them again, which needs a high cost.

But if you use web servers such as Apache APISIX, there will be no such a problem. Through API, you can easily modify in real time the upstream configuration of all machines in the entire cluster as well as the configuration of dynamic certificates. Though its performance will be slightly lower than Nginx, the marginally lower performance is acceptable because API brings higher flexibility. Now many manufacturers including Internet companies and some traditional companies are replacing Nginx with API.

2. Zero Trust Gateway

In the past two years, zero trust has been a popular concept in security field. In the traditional security field, we believe that border protection is very important, so we will use a firewall to check the incoming traffic. This check is actually a check of the hit rule. If the data is dangerous, it will be blocked. Then there exists a problem. If some rules are not updated in time, something dangerous can traverse the firewall. What’s worse, because we pay more attention to boundary protection, the insecure data can move freely within the internal network once it strides over the boundary.

But Zero Trust Gateway can completely solve this problem. It tacitly believes all traffic is insecure. It is completely based on identity authentication. For example, an API request will come to the identity authentication servers, through them the third-party identity authentication vendors such as Author0, OKTA will authenticate its identity. After the authentication, the request can pass, otherwise it will be rejected directly. This has become a trend in security field.

Enterprise Users’ Needs

  1. Unlocking and Supporting Backspace

The biggest need for all enterprise users is: don’t lock down users. API gateway is the entrance of the entire traffic, and users shouldn’t be locked in it. For example, if you use Alibaba Cloud’s API gateway, in fact, you have already been locked by it because it is the only entrance of the traffic. So, you can’t make distribution to other cloud vendors. In this situation, we need to backspace, which we have considered when we design Apache APISIX. For example, users may feel uncomfortable after using API gateway for a while and want to return to the previous interface. We can also meet this demand.

2. Keep the core stable

Apache APISIX’s iteration is very fast every month. There are about 10 main functions and more than 200 commits per month. However, within Apache APISIX, there is a directory called core, which has few changes. We will keep its core stable. That means, although functional iteration is fast, the core is extremely stable.

3. Support Enterprises to Develop Personalized Products

Whether domestic or foreign companies, there is no way to adapt products to all needs. If a product adapts to all the requirements, then it will become a giant. It will have complicated functions, low performance and low scalability. To solve this problem, we have plugins. If you need something personalized, you can customize your own plugins. If you need any feature, add this plugin to the system. You can also remove it if you don’t need it. The top layer can be customized according to users’ needs. Meanwhile, I can make sure that the bottom core layer is very stable.

How to Make Selection of Gateways

Industry Status Quo

  1. Refer to Gartner Report

We refer to the Gartner report when doing selection. We firstly explored the whole industry before we developed technology. As the picture above shows, top players such as Google, IBM, and RedHat have occupied most of the global API market. Besides, Kong, as an visionaries player, is known to the technology community. It plays a visionary role because it’s the first to make its own API Gateways open-sourced.

2. Refer to CNCF landscape

Gartner’s report is more commercialized. On this basis, we can refer to the landscape maintained by CNCF (Cloud Native Computing Foundation) belonging to the open source community. In the landscape, many open source and closed source projects are listed in layers and listed functionally. In the cloud native computing field, there are a dozen of softwares that can be selected when selecting API gateways. Half of them are products of major manufacturers, like 3SCALE (a product of Red Hat). The rest including Kong, Apache PISIX and TYK are all open source projects. We can choose among them when doing API selection.

Products Selection Comparison

After referring to the Gartner report and the CNCF landscape, we compared the mainstream API gateways and decided not to use the existing API gateways but to make one by ourselves.

  1. apigee

Apigee is the largest player, and its advantage lies in the lifecycle management. That is, the series of functions from API design, development, file management, testing, to launch, are all included in Google API gateway’s lifecycle management. However, apigee is a closed-source project. It can’t support customized development, and is locked by Google Cloud.

2. Kong

Kong solves the problems of apigee. It supports customized development but doesn’t lock down users. It has a fantastic idea to design its product and an accurate direction to develop. But there also exists a problem. Kong was developed in 2015, and since that time, the data has all been placed in a heavy relational database in Postgres, so it has complicated code and performance problems.

3. Apache APISIX

Different from Kong’s, Apache APISIX’s selection is based entirely on etcd. All data is placed in etcd to complete a lot of repetitive code that Kong has done on Postgres.

For example, message distribution, high availability and scalability are based on etcd, which makes operation easier. Although Apache APISIX has a shortcoming — short open source time (it was launched on June 6, 2019), it possesses an obvious advantage — its custom-made software development is much easier than Kong’s.

Take the example of rate and speed limiting function development, the function can be achieved through adding 60–70 lines of code to the Apache APISIX file. However, Kong needs to modify 5–6 files and 200–300 lines of code.

How to Make the Technological Selection of Gateways

API Gateways’ Core components

  1. Route. Route can be considered as various locations in Nginx. How to distribute locations to upstream servers and load various plugins is a function of route. Do we make a traversal or make a tree? Time required by them are different, so do the complexity. Apache APISIX using the Adaptive Radix Trees and Kong does this one by one. Obviously, Adaptive Radix Trees will be many times faster.
  2. Plugins. Plugin is one of API gateways’ core components. With plug-ins, we can easily develop our own code without waiting for the open source community to add related functions.When making plugins, we need to think about whether to make it hot-loaded. The significance of hot-loading is that when adding a new plugin or modifying a certain plugin, it doesn’t need to restart the entire service to take effect. We hope that when plugins are added or modified, they can take effect immediately through API calls.
  3. Schema verification. Schema verification refers to the verification on users’ requests. Fields and types uploaded by users must be verified whether they are legal or not. For example, when implementing a plugin, you need to verify whether its parameters and input values are valid. Schema is equivalent to the description of API, and it facilitates the cooperation between individuals and front-ends.
  4. Storing data in a relational database, a key-value database, or in etcd is equivalent to doing technical selection. You need to select these before you start.

Selection Principles: Be a friendly, high-performance, open source API Gateways under Cloud Native Computing

Be friendly to developers. It is the developers who use the API gateway. We hope that it will be easy for developers to watch the code, learn the code, and modify the plugins.

Pursue high performance. Although the performance of OpenResty is high, we find that the performance of the written code is not high when we actually write it. The code in OpenResty is easy to write, but it is difficult to make sure perfect code can be written. Because there are many traps that most people can hardly notice.

The Selection of Apache APISIX

  1. Route: Adaptive Radix Trees, which supports all Nginx vars, and custom function.
  2. Plugins: It is inspired by Kong, but the architecture and design of Apache APISIX plugins are completely different from Kong’s. It adopts hot-reloading and greatly simplifies the difficulty of code-writing.
  3. schema: jsonschema, which provides a JSON schema draft 4, draft 6, draft 7 validator for Lua/LuaJIT. Note that even though it uses the JSON Schema semantics, it is neither bound or limited to JSON. It can be used to validate saner key/value data formats as well (Lua tables, msgpack, bencode, …).
  4. Storage: Use etcd. We using Lua-resty-etcd to get configurations from etcd.

Apache APISIX’s Unique features

  1. Super performance. High performance is the main advantage of Apache APISIX. Through our tests, we found that the performance of Apache APISIX is 10 times as high as that of Kong.
  2. Hot-updated plugins. Plugins can be modified or added without restarting the service. All updates are hot.
  3. Support version contorl. If a newly-released version gets something wrong, you can easily backspace to the old version.
  4. Identity-based zero trust. The newly merged Future can support external identity authentication. At this time, it is easy to achieve zero trust and can access all services from external identity authentication vendors. We can directly add the identity authentication function to API after simply deploying some parameters.

The Best Practices of Testing and Continuous Integration

Testing

Large companies all have their own QA teams to do tests. The development department only needs to write the code and submit it to the QA team after a simple self-test. However, there is no QA team for open source projects, and even the development team members are part-time. Therefore, automated testing is a must and test-driven development should be adopted to implement open source projects.

OpenResty has nearly 70 open source projects, and its commercial companies have nearly 100 closed source projects. However, a total of nearly 170 projects are maintained by less than 10 people. If we do tests only by ourselves, we can’t do anything. So, we should find a way to solve this problem.

  1. Development is testing

It was slow at the beginning, because a corresponding test case also requires submitting after the development of a new function. Otherwise, PR will not be merged. For example, if you develop a function for OpenResty without submitting a corresponding test case or a complete test case. No matter how well the function is designed, the PR will not be merged, because you break the principle of the open source projects-tests should run automatedly.

Apache APISIX introduces an additional condition that code coverage cannot be lower than 70%, and now has been changed to no less than 80%. If your changes introduce new code and add test cases, but reduce the coverage of the original test cases, then the mergence is not allowed.

2. Unit tests are completely based on test::nginx
The test is completely based on test::nginx. It can be considered that test: Nginx is a DSL, with less documents and higher learning thresholds.

3. Code style detection: luackeck and lua-relang
Luackeck is very easy to use. It can be used in Lua and OpenResty. There are several parameters to choose from. You can also use lua-relang. We use both of them to check code style in order to ensure the consistency of code styles between ours and new committers.

4. Code coverage detection: luacov
We use luacov to check code coverage, which is a feature of standard lua, which can run the code coverage.
PR can be merged after all the above tests are completed successfully, namely, test: nginx, code style test, and code coverage test. Performance tests are performed when a new version is released on the 6th every month to compare the performance differences between the new and old versions. Generally, we use the flame graph and do fuzzing test regularly, and this test can randomly obfuscate the input values such as uri and args in order to simulate the situations under various stress tests. After all, Apache APISIX is an API gateway, as the entrance to the traffic, it must be stable.

5. Continuous Integration
Testing cannot always depend on people, otherwise one day it will not run. So how to make unit tests, performance tests, code style checks and other tests run independently of people?

Strong reliance on GitHub: issue, Milestone, code review, PR approved. GitHub’s code review is very easy to use. It can review every line of code. If it finds that there is no modification in the line of code which must be modified, there is no way to merge approved.

Strong reliance on travis CI: unit testing, code style detection, multi-platform testing (ubuntu and mac), front-end packaging, and automated submission. Travis is a continuously-integrated plugin for GitHub. Apache APISIX does not use its own server resources to run tests. Instead, it runs tests through the cloud. Now the packaging and submission of front-end are automated, and the compilation from vue to html is just done through travis.

coveralls.io: we upload the results of code coverage to the coveralls.io website, and visually presents everything related to code coverage, for example, which line of code has not been detected, and what has changed from the previous version. Through this way, code coverage is detected automatedly. As mentioned earlier, we must always ensure the stability of Apache APISIX’s core. Therefore, we need make sure that the code coverage of all files in the core directory can reach 100%. That is, every line of code in the core section must be under tests.

Conclusion

Less resources are not necessarily a bad thing. Regardless of whether it is an open source project or a company, only when there are few resources will you come up with some unique ideas or think about how to use external forces to solve problems.

It was through looking for “tricks” or adopting automated methods that we finished the selection, testing and CI of Apache APISIX. We took good advantage of external resources instead of creating another database only by ourselves.

The selection, testing, and CI of Apache APISIX are quite significant. In fact, they are more important than performance. Although we focus on performance, it is more worthwhile for everyone to consider not only the code, but CI and ways to test selection. You can use them in your business. Even without OpenResty and Apache APISIX, you can learn how to conduct open source projects.

Don’t waste energy on the services that GitHub and SaaS have provided. We should restrain the urge to “build the wheel”, that is, to develop softwares or databases that have existed.