nftfw logo

nftfw - Installation Guide

Prerequisites

This document assumes that you are installing on Debian Buster.

nftables

The standard version of nftables at the time of writing is: 0.9.0-2. Buster backports has a more recent version - 0.9.3-2~bpo10+1, and it's a good idea to upgrade to that. Look in /etc/apt/sources.list, and if necessary append:

then

which will upgrade your system to the most recent nftables release. The install action will tell you that a library is not needed before it installs things. There is a systemctl entry for nftables, that will probably be disabled now. Check with

I've got a live iptables installation

Well, what you need to do depends on what's running in your system. Buster comes ready installed with two flavours of iptables: the legacy version and a bridging version using the iptables API, which maps and loads the iptables formatted rules into nftables. You can tell which one you are using by:

if your version output looks like that, then you are OK and can just skip over what follows to Python.

If the words in brackets says legacy then you need to swap to the nf_tables version. Here's what you do:

FIrst, save the current iptables settings, you'll need to put the settings into the kernel's nftables system when you switch.

Now change iptables and ip6tables to use the nftables versions:

Each gives you a menu of options: select the nftables compatible version. I used option 0 - auto. Check your iptables command is now at the correct version using -V. Finally, reload the iptables data you saved earlier.

You now have two sets of rules loaded into the kernel, the new ones using nftables and the previous iptables set. The kernel will use the new rules but will complain. To remove the old rules:

All done, and it's painless. You have a system that works for both iptables commands and nftables nft command.

Python

nftfw is coded on Python 3 and the standard Python version on Buster is 3.7. The nftfw package was developed in part using Python 3.6, and so the nftfw package may run using that version.

We will use pip3 to install the nftfw package.

You may find that python3-setuptools and python3-wheel are already installed.

The nftfwls command uses Python's prettytable, which may not be installed:

 $ sudo apt install python3-prettytable

nftfw Installation

You now need the nftfw distribution. I put mine into /usr/local/src, it can be your home directory.

which will create an nftfw directory.

Now change into the nftfw directory and use pip3 to install the package. pip3 will allow you to uninstall the package at a later date, if you wish.

To uninstall, sudo pip3 uninstall nftfw.

pip3 installs four commands: nftfw, nftfwls, nftfwedit and nftfwadm in /usr/local/bin. Since these are system commands, they ought to be in /usr/local/sbin, but the Python installation system doesn't allow that.

Take a moment to see if the installation worked by asking nftfw for help:

The next step is to install the basic control files in /usr/local/etc/nftfw, the working directories in /usr/local/var/lib/nftfw, and the manual pages in /usr/local/share/man.

The Install.sh script will copy files from the distribution into their correct places. It asks several questions and permits you to control the installation phases. It's safe to run the script again, it will not replace the contents of any directory ending with .d, or the two control files in /usr/local/etc/nftfw. The script uses the standard system install program to do its work.

It's a good idea to make files in /usr/local/etc/nftfw owned by a non-root user, so they are easier to change without using sudo. For Symbiosis, the user should be admin, for Sympl it will sympl. The script asks for a user name and will create these files owned by that user. Later, it's important to edit /usr/local/etc/nftfw/config.ini to tell nftfw the user that you selected.

Take care, and slowly...

Answers for default installation:

  • Install under /usr/local? yes
  • See the files installed? your choice
  • Install? yes
  • User to replace root? 'admin' for Symbiosis, 'sympl' for Symbl, 'return' for root
  • Install Manual pages? yes

Alternatively, you can run the script without user interaction. Copy Autoinstall.default to Autoinstall.conf, edit the AUTO_USER to the user you want to use, and then run the script. Autoinstall.conf will be ignored by git, so this file can be used for later automated runs.

In /usr/local/etc/nftfw, you will find two files: config.ini and nftfw_init.nft. config.ini provides configuration information overriding coded-in settings in the scripts. All entries in the distributed files are commented out using a semi-colon at the start of the line. nftfw_init.nft is the framework template file for the firewall. It's copied into the build system whenever a nftfw creates a firewall. Also, you'll find the original directory holding all the original settings for the files. The intention is to provide a place for later updates to supply new and fixed default files.

install.sh also creates the necessary directories into /usr/local/var/lib/nftfw.

The final stage of the installation is to copy manual pages into /usr/local/share/man. There are six pages:

  • nftfw(1) - manage the Nftfw firewall generator. Describes the main command that creates and manages firewall tables.
  • nftfwls(1) - list the sqlite3 database used for storing IP addresses that have shown themselves to be candidates for blocking.
  • nftfwedit(1) - provides a command line interface to inspect IP addresses (both in and not in the blacklist database), and tools to add and delete IP addresses in the database, optionally adding them to the active blacklist.
  • nftfwadm(1) - provides some tools that may be useful when installing the system.
  • nftfw-config(5) - describes the contents of the ini-style config file tailoring settings in nftfw.
  • nftfw-files(5) - the format, names and contents of the files used to control the system.

The man command may need '5' in the command line to display the section 5 manual pages. Incidentally, the distribution also has these manual pages in HTML format (see docs/man).

As distributed, Debian Buster comes out of the box using nfttables as the basic firewall with a compatibility mode for iptables installed. Your system may vary. nftfw introduces specific nftables constructs, perhaps adding sets into the mix, the iptables interface will break down. You may have listed the kernel firewall with:

and now you need to re-educate yourself to run:

You are now ready to create firewall to suit your needs.

Paying attention to config.ini

Find the Owner section in the file and change settings for owner and group to fit the user you selected when installing the etc/nftfw files.

If you are running nftables and have a live /etc/nftables.conf file, you may need to alter the nftables_conf setting in config.ini and please read on. If not, skip to Logging.

Debian expects systems using nftables to keep a configuration file in /etc/nfttables.conf. The file sets up nftables when the system reboots, or when systemctl restarts the nftables service. nftfw will write this file after creating its rule set but depends on configuration in its config.ini file to set its location. As distributed, the value of nftables_conf in config.ini is relative to the installation root. This means you need to take different actions depending on where your nftfw is installed:

  • For nftfw installed in /usr/local: The default setting of nftables_conf will be /usr/local/etc/nftables.conf, which is the 'wrong' location, but is safe for now. You will eventually need to change the nftables_conf setting in config.ini to /etc/nftables.conf. Once you are happy with nftfw, you will then change the setting to its correct location in /etc.

  • For nftfw installed in /: The default setting of nftables_conf will be /etc/nftables.conf, which is the 'right' location, but maybe dangerous now. You probably should change the default setting in nftfw/config.ini to prevent it writing or installing /etc/nftables.conf until you are happy. Change the setting to place the file in perhaps /etc/nftfw/nftables.conf.new for now, and change it back later.

Logging

All nftfw programs will write logging message to syslog, and also to the terminal. Error messages are output using logging level ERROR, and information messages using INFO. The scripts turn off direct printing output unless they are talking to a terminal. The scripts all have a -q (quiet) flag suppressing terminal output.

The logging level displayed by the scripts is set by a value in the configuration file config.ini, and this defaults to ERROR, so only error messages are displayed. The scripts have a -v (verbose) flag that raises the output level to INFO, showing the information messages. Alternatively, the loglevel setting can be set in config.ini to always show these messages.

to

Using Symbiosis/Sympl control files

nftfw uses the same format for the control files found in the Symbiosis/Sympl firewall directory /etc/symbiosis/firewall.d or /etc/sympl/firewall.d. There is a setting in config.ini which tells nftfw where to look for these files, and this may point at the current directory. The five directories, incoming.d, outgoing.d, blacklist.d, whitelist.d and patterns.d are usable by nftfw.

nftfw makes three changes to the file format for patterns that make nftfw pattern files incompatible with those used by Symbiosis.

  • It's possible to set the ports= value in a pattern file to the word 'update'. The idea is to use the pattern scanning system to look in /var/log/syslog for messages logged by the firewall and use that information to update counts in the blacklist database. The 'update' action doesn't create a new record, it simply updates counts. A blacklisted site will often continue to hammer at the closed door, and the 'update' scan keeps the blacklisting from timing out until the site stops sending packets.

  • It's possible to set the ports= value in a pattern file to the word 'test'. See Testing regular expressions in the Users Guide.

  • It's possible to use shell-style 'glob' expressions in the file= statement, enabling the scanning of several related log files by the same set of regular expressions. This is useful for writing one set of rules for several websites whose log files are in separate files.

nftfw has its own set of rules used by actions to create control lines in the firewall. Symbiosis keeps its rules in /usr/share/lib/symbiosis/firewall/rule.d and nftfw has moved its files to its own directory in /usr/local/etc/nftfw/rule.d. Several of nftfw's rules are there to provide compatibility with Symbiosis and some of these return nothing because the functionality has migrated into the firewall framework provided by nftfw_init.nft.

Migrating to nftfw

The nftfw command builds the firewall, installs it in the kernel and saves a copy of what it has created in /etc/nftables.conf. Debian expects to reload nftables from the file in /etc on a reboot.

If your system has a running firewall that's not nftfw then you probably don't want to be too hasty about installing the system. I have no expectation that things will go wrong, but the ability to go back is important. The steps below provides ways to revert, and explain how to do the installation safely.

First, make sure you open another window to the system and login to the machine with it, this will keep the connection open and you can recover if something goes wrong.

Let's look at where you might be now:

  • The system runs a Symbiosis or Sympl system using iptables. In this case, the running firewall can be re-installed using the {symbiosis|sympl}-firewall command. You should move /etc/cron.d/symbiosis-firewall and /etc/incron.d/symbiosis-firewall (or equivalent sympl files if present) to a safe place to stop their firewalls from running for now.
  • Or: The system runs a different system using iptables, you can usually use iptables_save to save the settings somewhere, and iptables_restore to put the old rules back.
  • Or: Your system already runs an nftables based firewall, and you want to try nftfw out. In this case, do make sure that nftfw won't overwrite your /etc/nftables.conf file.
  • Or: There may be other options.

If you have a live iptables system, check the information about iptables versions and how to set things up at the top of this document.

Having set up the directories in /usr/local/etc/nftfw and the working directories in /usr/local/var/lib/nftfw, you can run:

The -x flag makes nftfw compile the files used in setting up the firewall, tests their syntax using the nft command, but doesn't install them. The -v flag prints information messages, and is a good idea if you've not altered logging levels in config.ini.

Assuming the tests show that the installation is OK, then you will have a working firewall to install. At this point, if you want to know what will happen, you can change to /usr/local/var/lib/nftfw/test.d and look at the files. The starting point for the firewall is nftfw_init.nft, which then includes further files. We cannot get a fully compiled set without loading the actual kernel tables, and you could do that if you are feeling confident, but not until you've taken the next step.

Installing

If you are on a Symbiosis or Sympl system, I recommend at this point that you move /etc/cron.d/{symbiosis|sympl}-firewall to a safe place. You don't want this firing when you are doing the next sequence of commands, so remove it from cron. On a Symbiosis system, you also need to move /etc/incron.d/symbiosis-firewall to a safe place, you don't want this starting Symbiosis if files in /etc/symbiosis/firewall.d change.

If you have a running nftables or iptables installation, now is the time to run:

this saves your nftables settings into nftfw's backup system, so if the install does fail, it will revert to what you had there before you started meddling. If you are using an iptables based system, fear not, the nft command doing the work will save the iptables settings in nftables format, assuming you have the nf_tables version of iptables installed and have working tables.

If all is well, you can try loading the rules made by nftfw.

The -f forces a full load and you'll need it if you've run the test. The point of saving the original settings will be apparent if something bad happens on this load, nftfw will reload the kernel state from your saved settings.

If there was no error, you can now see all the tables using:

If you need to revert, then now's the time.

replaces the newly installed rules with the tables you saved. The restore command will delete the backup file you stored, so you will need to run save again if you plan to change things and try again.

If you are happy with nftfw, then you are nearly all set. If you've used the save command to nftfwadm, then you need to remove the backup version of your old settings from the system.

simple deletes the backup file. Don't leave it installed, nftfw makes a backup file on every run so it can backtrack. However, it doesn't create a new file if it exists, and you need to remove your original settings from the system. If you don't remove the file and there's a problem some time in the future, you may find yourself wondering why the firewall in the system is an ancient version. Just where did that come from?

Final steps

To tidy up, check that the setting of nftables_conf in nftfw's config file /usr/local/etc/nftfw/config.ini reads:

and run

again to make sure that the new rules are written into /etc/nftables.conf.

Tell systemctl to enable and start its nftables service.

On a boot of a Symbiosis or Sympl system, the firewall starts at network up time and closes at network down time. Change into /etc/network and delete if-up.d/{symbiosis|sympl}-firewall and if-down.d/{symbiosis|sympl}-firewall}. The file is a symbolic link to the firewall script. This turns out to be an important step, rebooting without having this done results in a bad combination of two firewalls, because the nftables settings are loaded before the Symbiosis/Sympl ones.

Setting up cron

Like Symbiosis/Sympl, nftfw uses cron to drive regular polls by the firewall loader, and blacklist and whitelist scanners. You'll find a sample cron control file in the cronfiles directory in the distribution. Hopefully, by now, you've moved the Symbiosis or Sympl versions to somewhere where they are not active.

Check cron-nftfw in the cronfiles directory. The file contains /usr/local/ as a pathname, if your system is installed from root, you'll need to edit the file to point to the correct location.

Install cron-nftfw in /etc/cron.d/nftfw.

Making nftfw control directories 'active'

The original Symbiosis system used the incron system to make the nftfw control directories active, ensuring firewall updates when files alter, appear or disappear. Sympl stopped using incron because it's poorly maintained and buggy. Without automatic updates, the system needs reloading whenever a user changes the contents of one of the directories, and forgetting to run the reload command is a source of errors. nftfw will work happily without active directories, but it makes the life of the system admin easier.

nftfw supplies control files for systemd using its ability to track file changes and starting a firewall reload when a change is detected in one of the control directories. This system replaces the incron support supplied before version 0.6.

To install the files and start the system:

The last command should show that the unit is active.

You can test this by going to the whitelist.d directory and adding and removing 8.8.8.8 while monitoring /var/log/syslog. The nftfw command should run and update the firewall.

Finally, you can turn off incron, if it's running:

Finally a tip that's hard to find: reload systemd if you change the nftfw files after installation and starting:

Geolocation

The listing program nftfwls will print out the country that originated packets in the firewall using the geoip2 country database available from MaxMind. MaxMind don't charge but want you to create an account with them to access their files.

See Installing Geolocation.

If you want to use the blacknets feature of nftfw v0.7.0 and later to block countries, then geoip2 can be used to supply lists of IP networks.

See Getting CIDR lists.

Sympl users: Update your mail system after installation

A repository that steps through the changes I make to the standard exim4/dovecot systems on Sympl to improve feedback and detection of bad IPs - see Sympl mail system update.

You Are There

Now look at:

All documents can be found on the web from the nftfw website.

Acknowledgement

All of this is made possible by shamelessly borrowing ideas from Patrick Cherry who created the Symbiosis hosting package for Bytemark of which the firewall system is part.