% NFTFW-FILES(1) | Nftfw documentation
nftfw-files — documentation of file formats used in nftfw
This page documents the file formats used in the nftfw firewall system. The system stores various control files in /etc/nftfw or /usr/local/etc/nftfw depending on the installation.
The etc/nftfw directory contains:
Distributed files can be found in etc/nftfw/original.
Files in these directories specify rules for the firewall. File names have the format:
where 'number' is a pair of digits used as a sequence number and 'description' specifies the action name needed to created the nftables commands for the rule.
Descriptions can be:
When port numbers appear in the filename, the directory name dictates the action file applied for the rule. The config.ini file contains variables that select the default rule based on the directory name (see nftfw-config(1)).
To allow rules to have the same name as services and replace the default action, nftfw searches the rule directory for name matches before querying the service file.
Files are usually empty, but can contain a list of IP addresses (one per line) that nftfw uses to specify the source IP or IPs for an incoming rule, or the destination IP or IPs for the outbound rule. For example, supplying a list of known IP addresses for the standard ssh(1) service will prevent tiresome exhaustive attempts to get passwords. Local users can access ssh(1) from unknown addresses using the knowledge of a random port number given by another rule.
Files in these directories make nftables rules permitting access in the whitelist or blocking access in the blacklist. Whitelisted rules appear before blacklisted ones in the firewall.
Filenames are simply IP addresses. The whitelist or blacklist scanners will create files in these directories, and will add a suffix of .auto. Files added 'by hand' should just be the IP address.
IPv6 addresses are added in /64 form, with the '/' replaced by a vertical bar '|'. Install IPv4 address groups with network masking using the same convention.
Empty files mean that the rule applies to all ports. File contents are lists, one per line, with the following contents:
Firewall rules with 'all' ports appear in the ruleset before any rules containing specific ports.
The system has no way of distinguishing between TCP and UDP protocols and the system generates two rules for each rule it finds.
Administrators can disable the blacklist and whitelist systems separately by creating a file called 'disabled' in the relevant directory.
When building the firewall from these two directories, nftfw writes the IP addresses into nftables sets. The program writes the information into two separate files and uses file comparison with the last loaded files to see If it can update the sets of IP addresses without reloading the whole firewall.
Files in this directory make nftables rules in a single set that block ranges of IP addresses. To be included, a file must end in .nets.
Each file contains a list of IP network addresses, expressed in CIDR notation, one to a line. The file can also contain comments with the usual use of # to indicate them. Lines can contain the following formats:
# IPv4 CIDR 203.0.113.0/24
# IPv6 Compressed 2001:DB8::/32
# IPv6 Exploded 2001:0db8:0000:0000:0000:0000:0000:0000/32
# IPv4 embedded in IPv6 (will be converted to IPv4) # Format used by ip2location ::FFFF:203.0.113.0/120
# IPv4 embedded in IPv6 (will be converted to IPv4) ::ffff:cb00:7100/120
Theoretically, these are addresses of networks and not interfaces. The parts of the address that are 'local' should be zero. For example in 203.0.113.5/24, the '24' means that the leftmost 24 bits of the 32 bits are the network address and the 5 is the local part, that should be zero. However, nftfw will accept addresses where bits are set in these supposedly zero sections and will clear them.
The system can deal with lists for the same country from different sources, which will inevitably diverge. The system will remove exact duplicates, and will minimise overlapping address ranges where possible
Patterns define rules for the blacklist module containing the log file (or files) for scanning, the port numbers for the blocking firewall rules, and a list of regular expressions matching lines in the log file.
Pattern files are text files named name.pattern. The files support comments when the first character of the line contains '#'.
The files contain two 'equals' statements that should always be present:
file = filename ports = port specification
Filename is the full path to a logfile that the pattern will used to scan. The filename can also contain shell 'glob' characters ('*', '?' and single character ranges) allowing for the rule set to match a range of files. The blacklist system will ignore the pattern file (and complain) if the file (or files) that it nominates doesn't exist.
The port specification is usually a comma separated list of port numbers. A firewall rule uses the port list to ban access to specific services on the system. The ports statement has three 'special' values:
The remainder of the pattern file is a set of regular expressions, placed one per line, that match offending lines in the log files. The rules all contain the string
__IP__ (two underscores at end) used to match and capture the IPv4 or IPv6 address from the line. Non-empty lines that don't contain
__IP__ are flagged as errors.
The expressions support Python's standard regular expression syntax but must only have one matching 'capturing group' which is the
__IP__ expansion. It is safe to use non-capturing expressions, for example to match word1 or word2 in the line, use
Lines are flagged in the logs and ignored if the compilation of the regular expression fails, or if there is more than one matching group.
The blacklist action for nftfw uses the patterns to scan log files for matching lines using case-independent matching by the regex and finds IP addresses that it adds to an sqlite3(1) database. IP addresses exhibiting activity levels over a threshold will cause the script to add the IP address file to the blacklist directory (see nftfw(1)).
Setting ports=test in a pattern file enables testing to see if regular expressions pick up offending IP addresses. Set up a pattern test file pointing to the file you want to scan, and set port=test, add the regular expression you wish to test. Then running
sudo nftfw -x -p pattern-test blacklist
will use data from pattern-test.pattern and will scan the named log file (or files). The -x flag scans the log file from the beginning and will not update the stored file position. The command will print a table with any matching IP addresses, along with a match count. The command can be re-run if matches fail after adjusting the regular expression in the pattern file.
The rule directory contains small shell scripts that translate firewall actions named in the incoming.d and outgoing.d directories into nftables command lines. Default rules are also used for the whitelist and blacklist generation. Note the coding and management of these files are different from Symbiosis, but the same idea is there, a shell file allows easy additions by users. The files do not run any commands, they output nftables statements to nftfw which stores them and passes the file into the nft command.
Filenames have the format:
nftfw runs the scripts though the shell and captures the output text, appending it to an nftables command file. The system calls each action file twice, once for IPv4 and again for IPv6. The processing script uses environment variables to pass parameters into the shell. The parameters are:
DIRECTION - incoming | outgoing PROTO - values ip|ip6 TABLE - usually filter CHAIN - table to add the rule to PORTS - ports to use (can be empty) COUNTER - set to counter or empty IPS - ip addresses (can be empty, single, ranges, named sets, unnamed sets) LOGGER - logger statement
The pattern script uses the DIRECTION parameter in both incoming and outgoing contexts and must set directional keywords in nft commands correctly. For an incoming rule, an IP address (if present) will be a 'source' address. For an outgoing rule, an IP address (if present) will be a 'destination' address.
A rule script will usually create a simpler version of the command when called with no ports.
The etc directory contains the config.ini file for nftfw. nftfw_config(5) contains a description of the lines in this file.
The file nftfw_init.nft contains the template rule set for nftables, it's used to establish the firewall framework and finally uses several include statements to pull in the files created by the system. nftfw copies the file into the build directory at the start of the build process.
The file is user-editable, allowing the framework to be changed. The basic setup assumes that it's running on a system with a single network connection attached to the internet. See the example supplied in etc/nftfw/original/nftfw_router_example which provides a router setup with WAN and LAN connections using nat and forward tables.
Finally, the original directory contains the starting point for all control files, and some examples. The directory allows distributions to update the standard control files while not changing those in use on a running system.
The lib/nftfw directory provides working space for the system. It contains three directories and several working files.
Contents of build etc
Files can be located in /usr/local.
: Location of control files
See GitHub Issues: https://github.com/pcollinson/nftfw/issues
Peter Collinson (huge credit to the ideas from Patrick Cherry's work for the firewall for the Symbiosis hosting system).
nft(1), nftfw(1), nftfwls(1), nftfwadm(1), nftfw-config(5)