Implementing high-volume dynamic IP blacklists and whitelists in Nginx with Lua and Redis in Ubuntu and Debian

dynamic IP blacklists and whitelists in Nginx
Андрей Васенин

Андрей Васенин

Автор статьи. Сфера интересов: ИТ-специалист (программирование, администрирование, DBA). Кандидат экономических наук. Подробнее .

In this tutorial, we will look at how to implement the concept of a large dynamic blacklist or whitelist using an Nginx web server, a Redis server, and an Nginx extension script implemented in Lua.

In this tutorial, all software will be installed on Ubuntu 16.04 or 18.04. With little or no change, these solutions can be applied to other Debian-based distributions.

Nginx has a built-in blacklisting or whitelisting mechanism that is suitable for small lists. However, this mechanism is inconvenient when you need to work with large lists of addresses, when the lists are generated dynamically.

Fortunately, Nginx supports Lua plugins that, together with the Redis in-memory cache, allow you to implement the required whitelisting and blacklisting functionality.

Installing Nginx and Redis on Ubuntu Linux

All components will be installed using APT:

$ sudo apt update
$ sudo apt install redis-server nginx nginx-extras lua-nginx-redis

We check the availability of Nginx by opening the page on the server ip in the browser:

check the availability of Nginx by opening the page

Let's check the availability of Redis:

$ redis-cli> ping
PONG> quit

If you're setting up a server for production use, be sure to set up Redis security with our guide .

Let's move on to setting up a white or black list.

Setting up a whitelist

We'll be using a recipe for a whitelist script that will lead to GitHub . To use it, let's add several directives to Nginx:

Add the following line to the /etc/nginx/nginx.conf file:

http {
  # Basic Settings
  lua_shared_dict ip_whitelist 1m;

and in the server or location section, the whitelisted handler itself. In our example, add to the /etc/nginx/sites-enabled/default file:

# Default server configuration
server {
     listen 80 default_server;
     listen [::]:80 default_server;
     access_by_lua_file /etc/nginx/lua/ip_whitelist.lua;

Download the Lua script itself:

$ sudo mkdir /etc/nginx/lua
$ cd /etc/nginx/lua
$ sudo wget

Let's check the Nginx configuration and restart it:

$ sudo nginx -t && sudo service nginx restart

If you now try to open the site, you will get a page with an error like:

Access denied

Now add your IP to Redis using the Redis console:

$ redis-cli SADD ip_whitelist A.B.C.D

and check the site access again, the Nginx page we saw before should open:

check the availability of Nginx by opening the page

Again, remove the entry for your IP using the Redis console:

$ redis-cli SREM ip_whitelist

The result of accessing the site will again be an error.

Thus, we have successfully set up a whitelist using Nginx, Redis, and a Lua script that is suitable for dynamically managing entries and working with large lists. The list is implemented using high-performance Redis sets, the access speed to which obeys the O (1) labor law, that is, it works quickly if in Russian.

Blacklist setup

The black list is configured in the same way, with an accuracy of replacing white with black :

  • lua_shared_dict ip_blacklist 1m;
  • access_by_lua_file /etc/nginx/lua/ip_blacklist.lua;
  • script .

Modify the script to use the correct Redis server and the correct Lua module for it (the author uses resty.redis for this script, not nginx.redis):

$ sudo sed -i 's|||' ip_blacklist.lua
$ sudo sed -i 's| \= require "resty.redis"| \= require "nginx.redis"|' ip_blacklist.lua 

Adding and removing entries to the blacklist is carried out by the following commands:

$ redis-cli SADD ip_blacklist A.B.C.D
$ redis-cli SREM ip_blacklist A.B.C.D


Dynamic black and white lists are a more versatile solution when you need to use large lists with the need to change records on the fly, which may be useful if you have intruder diagnostics.

Storing lists in Redis allows you to provide the highest solution performance that does not degrade as the list of addresses in the list grows. API availability of Redis is suitable for integration with third-party list record providers.

In the event that it is necessary to preserve the lists between Redis reboots, then the line in the /etc/redis/redis.conf configuration file must be changed:

appendonly no
appendonly yes

The Lua script is quite simple and can be easily adapted for more specific tasks.

Вас заинтересует / Intresting for you:

What Is Bitcoin? Deep descript...
What Is Bitcoin? Deep descript... 2288 views Максим Николенко Tue, 26 Nov 2019, 05:02:43
How to Subnet IPv4 Addresses? ...
How to Subnet IPv4 Addresses? ... 824 views Валерий Павлюков Sat, 05 Feb 2022, 05:44:19
What are IPv4 addresses used f...
What are IPv4 addresses used f... 1079 views Андрей Волков Sat, 13 Feb 2021, 18:42:13
Cisco Cables and Connections d...
Cisco Cables and Connections d... 511 views Валерий Павлюков Sat, 05 Feb 2022, 17:34:37
Comments (0)
There are no comments posted here yet
Leave your comments
Posting as Guest
Suggested Locations