Suricata +
Integrating AbuseIPDB with Suricata - Automatically Block Bad IPs

AbuseIPDB provides a free API for reporting and checking IP addresses. Every day webmasters, system administrators, and other IT professionals use our API to report thousands of IP addresses engaging spamming, hacking, vulnerability scanning, and other malicious activity in real time.

Pre-Requisites - Before You Start This Tutorial

1) Install Suricata on your server

Before starting this tutorial, we assume that you have a Linux server with Suricata installed. A tutorial for installation is available on the Suricata read the docs we also highly advise installing the dependencies recommended on this page, as it make this tutorial much smoother for you.

2) Create an AbuseIPDB API key

We also assume that you have an account registered with AbuseIPDB, and have verified your domain and created an API key. The API is free to use, but you do have to create an account.

Basic Setup

3) Install Lua

Lua package is available in official repositories of major Linux distributions, you can install the latest version using the appropriate package manager on your system. For RHEL/CentOS run the following commands:

yum install epel-release
yum install lua

Now check to see that you have the most recent version of Lua by running:

lua -v

Suricata currently supports Lua 5.1 and not Lua 5.3 Therefore, you should have version 5.1.x installed to continue.

4) Install LuaRocks and the necessary packages

LuaRocks is the package manager for Lua. It can be installed by entering the following commands into your terminal:

wget https://luarocks.org/releases/luarocks-3.1.3.tar.gz
tar zxpf luarocks-3.1.3.tar.gz
cd luarocks-3.1.3
./configure; sudo make bootstrap

Now that LuaRocks is installed, let's install luasocket by typing the following command into your terminal:

sudo luarocks install luasocket

We can verify that it was installed correctly by running:

luarocks list

your output should look like this:

luarocks
	3.1.3-1 (installed) - /usr/local/lib/luarocks/rocks-5.1

The following packages should also be installed in the same manner:

lua-cjson
lua-requests
luafilesystem
luasec
lub
md5
xml

Run the following command to verify that you have the required packages:

luarocks list

Your output should be close to:

Rocks installed for Lua 5.1
	lua-cjson
			2.1.0-1 (installed) - /usr/local/lib/luarocks/rocks-5.1
	lua-requests
			1.2-1 (installed) - /usr/local/lib/luarocks/rocks-5.1
	luafilesystem
			1.7.0-2 (installed) - /usr/local/lib/luarocks/rocks-5.1
	luarocks
			3.1.3-1 (installed) - /usr/local/lib/luarocks/rocks-5.1
	luasec
			0.8-1 (installed) - /usr/local/lib/luarocks/rocks-5.1
	luasocket
			3.0rc1-2 (installed) - /usr/local/lib/luarocks/rocks-5.1
	lub
			1.1.0-1 (installed) - /usr/local/lib/luarocks/rocks-5.1
	md5
			1.3-1 (installed) - /usr/local/lib/luarocks/rocks-5.1
	xml
			1.1.3-1 (installed) - /usr/local/lib/luarocks/rocks-5.1

Configuration Setup

5) Suricata.yaml

Open the main configuration file suricata.yaml in your favorite text editor. The following command will open the file in nano:

nano /etc/suricata/suricata.yaml

This file is actually already organized into a step-by-step process, but we will help guide you through it as well.

Navigate to line 14, the address-groups list the definitions for HOME_NET and EXTERNAL_NET.
If need be, uncomment the fourth occurrence of HOME_NET, this should be on line 18.
Add IPs here that you want as part of your HOME_NET, or home network.

Navigate to the first occurrence of EXTERNAL_NET, this should be on line 21. Ensure that the line is uncommented, and says: EXTERNALNET: "!$HOME_NET"

The next step is to read over the "Step 2" section of the file, and decide which outputs you want enabled. For reference, we have the following enabled on our suricata.yaml:

lua
stats
fast
eve-log
alert
http
dns
http-log
tls-log
pcap-log
alert-debug
drop
tcp-data
http-body-data

In order to use the script to report to AbuseIPDB, you MUST enable the lua output module. For us, the lua section starts on line 566. Under scripts: add abuseipdb_report.lua. Your Lua section should be as follows:

- lua:
enabled: yes
 scripts-dir: /etc/suricata/lua-output
 scripts:
  - abuseipdb_report.lua

To ensure that updating the rules will go smoothly, let's also change the default-rule-path and rule-files options to the following:

default-rule-path: /var/lib/suricata/rules
rule-files:
 - suricata.rules
 - local.rules

Save the file before closing.

6) disable.conf

Some of the rules include in the Emerging Threats, and other open source rulesets are just too strict for standard operations. For this reason, and based on other recommendations found on their forums, we have compiled a list of rules that we think should be disabled. Open the file disable.conf with your favorite text editor. If the file is blank, there is no need to worry. The following command opens the file in nano:

nano /etc/suricata/disable.conf

Copy and Paste the following into your disable.conf:

2010494 #disable this sid
2200074 #disable this sid
2210059 #disable this sid
2001219 #disable this sid

# disable rules containing the following:
re:invalid checksum
re:invalid timestamp
re:package management
re:SSH Scan
re:3way handshake
re:SURICATA STREAM
re:SURICATA HTTP
re:Applayer Wrong direction first Data
re:Mismatch protocol both directions
re:protocol only one direction
re:header length too small
re:invalid handshake message

Save the file before closing.

Setting up the Lua script

Below is the Lua script that we developed to report to AbuseIPDB. Please use this script and replace $YOUR_API_KEY with your API key and leave it surrounded with qutation.

Once the script is dowloaded, enter the following commands into your terminal:

cd /etc/suricata/
mkdir lua-output

Move the script into the /etc/suricata/lua-output directory that you just made. This is the default directory where Suricata will look for Lua output scripts to execute. Change the permissions of the script to be an executable using:

chmod +x abuseipdb_report.lua

Creating the Whitelist

Suricata allows you to add your own custom rules. The pass action allows you to choose which IP(s) you would like to trust. Open the file local.rules in your favorite text editor. We used nano:

nano /var/lib/suricata/rules/local.rules

Copy and Paste the following into your local.rules file:

 pass ip $HOME_NET any -> any any (msg:"pass all traffic from Home Network"; sid:1000001;)

Managing the rules

We used suricata-update to manage our rules foe Suricata. Run the following command in the terminal anytime you would like to update your Suricata rules:

sudo suricata-update

This updates the rules based on the disble.conf and enable.conf files and also downloads the Emerging Threats Open ruleset.

Adding more rulesets

To see a list of the available sources, run the following commands in the terminal:

sudo suricata-update update-sources
sudo suricata-update list-sources

The first command ensures the list is up to date, and the second displays the list. Each of the rulesets has a name that has a "vendor" prefix, followed by a set name. For example, OISF’s traffic id ruleset is called "oisf/trafficid".

To enable "oisf/trafficid" enter:

sudo suricata-update enable-source oisf/trafficid
sudo suricata-update

Enable the following other rulesets in the same manner:

sslbl/ssl-fp-blacklist
ptresearch/attackdetection

Run the following command into your terminal:

sudo suricata-update list-enabled-sources

Make sure the end of your output contains the following:

Enabled sources:
		- oisf/trafficid
		- et/open
		- sslbl/ssl-fp-blacklist
		- ptresearch/attackdetection

Run suricata-update one last time.

Restart script

For Your convenience, we have written a bash script to restart Suricata in one command. We have placed it below.

The only thing needed is to change the permissions by running:

chmod +x restartSuricata.sh

That's it! You can use that script to restart Suricata anytime you would like by running:

./restartSuricata

Optional integration with UncomplicatedFirewall (UFW)

We have written two scripts for you, one to block any IPs reported via Suricata on UFW and one to unblock those IPs. Please follow the below tutorial to set these scripts up for use on your server that uses UFW as a firewall. If you do not have UFW installed, please do so before continuing.

Create the file ufw_block.py

Either through a text editor or the terminal, enter the following code into the file and save it.

import os
from datetime import date

blockedFile = open("/var/log/suricata/blocked_ips.log", "w")
blockList = []
with open('/var/log/suricata/reported_ips.log') as file:
    for line in file:
        blockedFile.write(line)
        lineArray = line.split()
        ip = lineArray[0]
        blockList.append(ip)
        os.system("sudo ufw deny from " + str(ip))

blockedFile.close()
print("the following IPs have been blocked on ufw: ")
for ip in blockList:
    print(ip)

file = open('reported_ips.log', 'w')
file.close()

Create the file ufw_unblock.py
Either through a text editor or the terminal, enter the following code into the file and save it.

import os
from datetime import datetime

output = []

with open('/var/log/suricata/blocked_ips.log') as file:
    for line in file:
        lineArray = line.split()
        dateArray = lineArray[1].split("-")
        ip = lineArray[0]
        reportDate = dateArray[0]
        reportDate = datetime.strptime(reportDate, "%m/%d/%Y")
        today = datetime.today()

        # Get the difference in days
        difference = abs((today - reportDate).days)
        # If it has been a week or more, unblock the ip from ufw
        if difference >= 7:
            os.system("sudo ufw delete deny from " + str(ip))
        # Otherwise, add the line to the output list to write to the file
        else:
            output.append(line)

# Remove the lines that have been unblocked by writing over with the ones that were not
file = open('blocked_ips.log', 'w')
file.writelines(output)
file.close()

Modify the abuseipdb_report.lua script located in /etc/suricata/lua-output/

Locate the line file2 = assert(io.open(filename2, "a")) which should be line 19
Add the following code below that line:

filename3 = SCLogPath() .. "/" .. "reported_ips.log"
file3 = assert(io.open(filename3, "a"))

Locate the line file:write ("\nReport " .. count .. " Success! " .. comment .. " " .. table.concat(response_body)) which should be around line 90
Add the following code below that line:

file3:write(srcip .. " " .. timestring .. "\n")

Locate the line io.close(file2) towards the very end of the file.
Add the following code below that line:

io.close(file3)

Restart Suricata using ./restartSuricata to apply the changes in the script

NOTE: If your reportd_ips.log file is blank, you may need to restart Suricata again once it is running for a while since doing so makes Suricata write to its log files.

Use either script with python3 followed by the script name. For example:

python3 ufw_block.py

It is recommended to run ufw_block.py then ufw_unblock.py in that order

Thanks for supporting AbuseIPDB! Do you have any feedback or suggestions about this tutorial? Please let us know!

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF and its supporting vendors.

Suricata + Integrating AbuseIPDB with Suricata - Automatically Block Bad IPs