Tag Archives: unix

CloudFlare API v4 and Fail2ban: Fixing the unban action

This posting is ~2 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

In January 2017, I wrote an article about how to protect your WordPress blog using the WP Fail2Ban plugin, fail2ban on your Linux/ FreeBSD host, and CloudFlare. Back then, the fail2ban was using the CloudFlare API V1, which was already deprecated since November 2016.

Free-Photos/ pixabay.com/ Creative Commons CC0

Although the actions were updated later to use CloudFlare API V4, I still had problems with the unbaning of IP addresses. IP addresses were banned, but the unban action failed. 

This is the unban action, which is included in fail2ban (taken from fail2ban-0.10.3.1 which is shipped with FreeBSD 11.1-RELEASE-p10):

actionunban = curl -s -o /dev/null -X DELETE -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
            https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(curl -s -X GET -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
            'https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1' | cut -d'"' -f6)

And this is the unban action, which finally solved this issue:

actionunban = curl -s -o /dev/null -X DELETE -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
            https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(curl -s -X GET -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
            'https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1' | tr -d '\n' | cut -d'"' -f6)

I found the solution at serverfault.com. The only difference is an additional tr -d ‘\n’  in the last line of the statement. Kudos to Jake for fixing this!

To prevent the action file to being overwritten, you should copy the original cloudflare.conf  located in the  action.d  directory, e.g. to mycloudflare.conf , and use the copied action file in your fail definition.

[wordpress]
enabled  = true
filter   = wordpress
logpath  = /var/log/messages
action   = mycloudflare

Simplemonitor – Python-based monitoring

This posting is ~3 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

While searching for a simple monitoring für my root servers, I’m stumbled over a python-based software called Simplemonitor. Other alternatives, like Nagios, or forks like Incinga etc., were a bit too much for my needs.

What is SimpleMonitor?

SimpleMonitor is a Python script which monitors hosts and network connectivity. It is designed to be quick and easy to set up and lacks complex features that can make things like Nagios, OpenNMS and Zenoss overkill for a small business or home network. Remote monitor instances can send their results back to a central location.

My requirements were simple:

  • Ping monitoring
  • TCP monitoring
  • HTTP monitoring
  • Service monitoring
  • Disk space monitoring

Monitoring is nothing without alerting, so I was pretty happy that Simplemonitor is able to send messages into a Slack channel! But it can also send e-mails, SMS, or it can write into a log file. To get a full feature overview, visit the Simplemonitor website.

The project is hosted on GitHub. If you are familiar with Python, you can contribute to the project, or you can add features as you need.

Installation & configuration

The installation is pretty simple: Just fetch the ZIP or the tarball from the project website, and extract it.

The configuration is split into two files:

  • monitor.ini
  • monitors.ini

The naming is a bit confusing. The monitor.ini contains the basic monitoring configuration, like the interval for the checks, the alerting and reporting settings. The monitors.ini contains the configuration of the service checks. That’s confusing, that confused me, and so I changed the name of the monitors.ini to services.ini.

[monitor]
interval=60
monitors=services.ini

The services.ini (monitors.ini) contains the service checks. This is a short example of a ping, a service check, a port check, and a disk space check.

[ping-host1]
type=host
host=host1.tld.de
tolerance=3

[svc-postfix-host1]
type=rc
runon=host1.tld.de
service=postfix

[port-postfix-host1]
type=tcp
host=host1.tld.de
port=25

[diskspace]
type=diskspace
partition=/
limit=4096M

The alerting is configured in the monitor.ini. I’m using only the Slack notification. All you need is a web hook and the corresponding web hook URL.

[slack]
type=slack
channel=#monitoring
limit=1
url=https://hooks.slack.com/services/afjnsdifnsdfnsdf

In case of a service fail, or service recovery, a notification is sent to the configured Slack channel.

To start Simplemonitor, just start the monitor.py. It expects the monitor.ini in the same directory.

[email protected] /opt/simplemonitor # python2 monitor.py -v
SimpleMonitor v1.7
--> Loading main config from monitor.ini
--> Loading monitor config from services.ini
Adding host monitor ping-host2
Adding rc monitor svc-postfix-host1
Adding rc monitor svc-nginx-host1
Adding rc monitor svc-mysql-host1
Adding rc monitor svc-fail2ban-host1
Adding rc monitor svc-postgrey-host1
Adding rc monitor svc-phpfpm-host1
Adding rc monitor svc-named-host1
Adding diskspace monitor diskspace
--> Loaded 9 monitors.

Adding logfile logger logfile
Adding slack alerter slack

--> Starting... (loop runs every 60s) Hit ^C to stop
php_fpm is running as pid 33937.
Passed: svc-phpfpm-host1
named is running as pid 566.
Passed: svc-named-host1
fail2ban is running as pid 41306.
Passed: svc-fail2ban-host1
Passed: diskspace
postgrey is running as pid 649.
Passed: svc-postgrey-host1
mysql is running as pid 23726.
Passed: svc-mysql-host1
Passed: ping-host2
postfix is running as pid 53332.
Passed: svc-postfix-host1
nginx is running as pid 52736.
Passed: svc-nginx-host1

Summary

I really like the simplicity of Simplemonitor. Download, extract, configure, run, done. That’s what I’ve searched for. It is still under development, but you should not expect that it will gain much complexity. Even if features will be added, it should be a simple monitoring.

Stunnel and Squid on FreeBSD 11

This posting is ~3 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

I don’t like to use untrusted networks. When I have to use such a network, e.g. an open WiFi network, I use a TLS encrypted tunnel connection to encrypt all web traffic that travels through the untrusted network. I’m using a simple stunnel/ Squid setup for this. My setup consists of three components:

  • Stunnel (server mode)
  • Squid proxy
  • Stunnel (client mode)

What is stunnel?

Stunnel is an OSS project that uses OpenSSL to encrypt traffic. The website describes Stunnel as follows:

Stunnel is a proxy designed to add TLS encryption functionality to existing clients and servers without any changes in the programs’ code. Its architecture is optimized for security, portability, and scalability (including load-balancing), making it suitable for large deployments.

How it works

The traffic flow looks like this:

Stunnel Secure Tunnel Connection Diagram

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

The browser connects to the Stunnel client on 127.0.0.1:8080. This is done by configuring 127.0.0.1:8080 as proxy server in the browser. The traffic enters the tunnel on the client-side, and Stunnel opens a connection to the server-side. You can use any port, as long as it is unused on the server-side. I use 443/tcp. The connection is encrypted using TLS, and the connection is authenticated by a pre-shared key (PSK). On the server, the traffic leaves the tunnel, and the connection attempt of the client is directed to the Squid proxy, which listens on 127.0.0.1:3128 for connections. Summarized, my browser connectes the Squid proxy on my FreeBSD host over a TLS encrypted connection.

Installation and configuration on FreeBSD

Stunnel and Squid can be installed using pkg install .

[email protected]:~ # pkg search squid-3.5
squid-3.5.24_2                 HTTP Caching Proxy
[email protected]:~ # pkg search stunnel
stunnel-5.41,1                 SSL encryption wrapper for standard network daemons

The configuration files are located under /usr/local/etc/stunnel and /usr/local/etc/squid. After the installation of stunnel, an additional directory for the PID file must be created. Stunnel is not running with root privileges, thus it can’t create its PID file in /var/run.

[email protected]:/var/run # mkdir /var/run/stunnel/
[email protected]:/var/run # chown stunnel:stunnel /var/run/stunnel

The stunnel.conf is pretty simple. I’m using a Let’s Encrypt certificate on the server-side. If you like, you can create your own certificate using OpenSSL. But I prefer Let’s Encrypt.

cert = /usr/local/etc/letsencrypt/live/server/fullchain.pem
key = /usr/local/etc/letsencrypt/live/server/privkey.pem
pid = /var/run/stunnel/stunnel.pid
setuid = stunnel
setgid = stunnel
sslVersion = TLSv1.2
debug = 3
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
verify = 2
compression = deflate

[Tunnel.Web]
accept = 46.x.x.x:443
connect = 127.0.0.1:8080
ciphers = PSK
PSKsecrets = /usr/local/etc/stunnel/psk.txt
CAFile = /usr/local/etc/letsencrypt/live/server/fullchain.pem

The psk.txt contains the pre-shared key. The same file must be located on the client-side. The file itself it pretty simple – username:passphrase. Make sure that the PSK file is not group- and world-readable!

patrick:SuperSecretPassw0rd

The squid.conf is also pretty simple. Make sure that Squid only listens on localhost! I disabled the access log. I simply don’t need it, because I’m the only user. And I don’t have to rotate another logfile. Some ACLs of Squid are now implicitly active. There is no need to configure localhsot or 127.0.0.1 as a source, if you want to allow http access only from localhost. Make sure, that all requests are only allowed from localhost!

acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl Safe_ports port 2222
acl Safe_ports port 8443
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access deny all
icp_access deny all
htcp_access deny all
http_port 127.0.0.1:8080
cache_mem 1024 MB
maximum_object_size_in_memory 8 MB
cache_dir ufs /var/squid/cache 1024 16 256 no-store
minimum_object_size 0 KB
maximum_object_size 8192 KB
cache_swap_low 95
cache_swap_high 98
logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
# access_log /var/log/squid/access.log combined
access_log none
cache_log /dev/null
cache_store_log /dev/null
ftp_user joe.doe@gmail.com
htcp_port 0
coredump_dir /var/squid/cache
visible_hostname proxy

To enable stunnel and squid in the /etc/rc.conf, add the following lines to your /etc/rc.conf. The stunnel_pidfile  option tells Stunnel, where it should create its PID file.

squid_enable="YES"
stunnel_enable="YES"
stunnel_pidfile="/var/run/stunnel/stunnel.pid"

Make sure that you have initialized the Squid cache dir, before you start squid. Initialize the cache dir, and start Squid and Stunnel on the server-side.

Installation and configuration on Windows

On the client-side, you have to install Stunnel. You can fine installer files for Windows on stunnel.org. The config of the client is pretty simple. The psk.txt contain the same username and passphrase as on the server-side. The file must be located in the same directory as the stunnel.conf on the client.

socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
client = yes
sslVersion = TLSv1.2
debug=2

[Tunnel.Web]
accept = localhost:8080
connect = 46.x.x.x:443
PSKsecrets = psk.txt

Test your connection

Start Stunnel on your client and configure 127.0.0.1:8080 as proxy in your browser. If you access https://www.whatismyip.com, you should see the IP address of your server, not the IP address of your local internet connection.

You can check the encrypted connection with Wireshark on the client-side, or with tcpdump on the server-side.

Please note, that the connection is only encrypted until it hits your server. Traffic that leaves your server, e.g. HTTP requests, are unencrypted. It is only an encrypted connection to your proxy, not and encrypted end-2-end connection.