Tag Archives: freebsd

Using Let’s Encrypt DNS-01 challenge validation with local BIND instance

This posting is ~5 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

I’m using Let’s Encrypt certificates for a while now. In the past, I used the standalone plugin (TLS-SNI-01) to get or renew my certificates. But now I switched to the DNS plugin. I run my own name servers with BIND, so it was a very low hanging fruit to get this plugin to work.

Clker-Free-Vector-Images/ pixabay.com/ Creative Commons CC0

To get or renew a certificate, you need to provide some kind of proof that you are requesting the certificate for a domain that is under your control. No certificate authority (CA) wants to be the CA, that hands you out a certificate for google.com or amazon.com…

The DNS-01 challenge uses TXT records in order to validate your ownership over a certain domain. During the challenge, the Automatic Certificate Management Environment (ACME) server of Let’s Encrypt will give you a value that uniquely identifies the challenge. This value has to be added with a TXT record to the zone of the domain for which you are requesting a certificate. The record will look like this:

_acme-challenge.example.com. 300 IN TXT "ghd63jkcchaow92334...3kahgm9d872"

This record is for a wildcard certificate. If you want to get a certificate for a host, you can add one or more TXT records like this:

_acme-challenge.mx.example.com. 300 IN TXT "ghd63jkcchaow92334...3kahgm9d872"
_acme-challenge.www.example.com. 300 IN TXT "kauezwhcn745njsf....adowerß22"
_acme-challenge.example.com. 300 IN TXT "uqiwo97634bsncös....90237j2k812"

There is a IETF draft about the ACME protocol. Pretty interesting read!

Configure BIND for DNS-01 challenges

I run my own name servers with BIND on FreeBSD. The plugin for certbot automates the whole DNS-01 challenge process by creating, and subsequently removing, the necessary TXT records from the zone file using RFC 2136 dynamic updates.

First of all, we need a new TSIG (Transaction SIGnature) key. This key is used to authorize the updates.

root@vps ~ # dnssec-keygen -a HMAC-SHA512 -b 512 -n HOST letsencrypt
Kletsencrypt.+165+15583
root@vps ~ #

This key has to be added to the named.conf. The key is in the .key file.

key "letsencrypts" {
  algorithm hmac-sha512;
  secret "Q+NqA3DJR\5ü77nQ6r//+5QyPKeOyxPD==n09qb516>CTqX+BoG1BeR/9BIEº2 ff4RrDKky4jJ3FJWnQD3nqiJ<J";
};

The key is used to authroize the update of certain records. To allow the update of TXT records, which are needed for the challenge, add this to the zone part of you named.con.

zone "example.com" in {
        type                    master;
        file                    "/usr/local/etc/namedb/master/example.com.zone";
        allow-transfer          { xxxx:yyyy:z:aaaa::1; 123.123.123.1; };
        allow-query             { any; };
        also-notify             { xxxx:yyyy:z:aaaa::1; 123.123.123.1; };
        update-policy           {
            grant letsencrypt  name _acme-challenge.example.com. txt;
            grant letsencrypt  name _acme-challenge.www.example.com. txt;
            grant letsencrypt  name _acme-challenge.mail.example.com. txt;
        };
};

The records start always with _acme-challenge.domainname.

Now you need to create a config file for the RFC2136 plugin. This file also includes the key, but also the IP of the name server. If the name server is running on the same server as the DNS-01 challenge, you can use 127.0.0.1 as name server address.

dns_rfc2136_server = 127.0.0.1
dns_rfc2136_name = letsencrypt
dns_rfc2136_secret = Q+NqA3DJR\5ü77nQ6r//+5QyPKeOyxPD==n09qb516>CTqX+BoG1BeR/9BIEº2 ff4RrDKky4jJ3FJWnQD3nqiJ<J
dns_rfc2136_algorithm = HMAC-SHA512

Now we have everything in place. This is a –dry-run  from on of my FreeBSD machines.

root@vps ~ # certbot renew --dry-run --dns-rfc2136 --dns-rfc2136-credentials /root/rfc2136.ini --server https://acme-v02.api.letsencrypt.org/directory --dns-rfc2136-propagation-seconds 5
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /usr/local/etc/letsencrypt/renewal/host.example.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator dns-rfc2136, Installer None
Renewing an existing certificate
Performing the following challenges:
dns-01 challenge for host.example.com
Waiting 5 seconds for DNS changes to propagate
Waiting for verification...
Cleaning up challenges

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/usr/local/etc/letsencrypt/live/host.example.com/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
  /usr/local/etc/letsencrypt/live/host.example.com/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
root@vps ~ #

This is a snippet from the name server log file at the time of the challenge.

27-Oct-2018 17:54:52.249 update: info: client @0x8031d8a00 127.0.0.1#34120/key letsencrypt: updating zone 'example.com/IN': adding an RR at '_acme-challenge.host.example.com' TXT "QE6ow9YttB580nKw5jgRTOo(nDû3e3I_Ñ2-)j-rY951"
27-Oct-2018 17:54:52.259 notify: info: zone example.com/IN: sending notifies (serial 2018061426)
27-Oct-2018 17:55:00.787 update: info: client @0x804011000 127.0.0.1#34121/key letsencrypt: updating zone 'example.com/IN': deleting an RR at _acme-challenge.host.example.com TXT
27-Oct-2018 17:55:00.810 notify: info: zone example.com/IN: sending notifies (serial 2018061427)

You might need to modify the permissons for the directory which contains the zone files. Usually the name server is not running as root. In my case, I had to grant write permissions for the “bind” group. Otherwise you might get “permission denied”.

26-Oct-2018 11:24:17.215 update: info: client @0x8031d8a00 127.0.0.1#41221/key letsencrypt: updating zone 'example.com/IN': adding an RR at '_acme-challenge.example.com' TXT "bmy-c8L8AOykzMHi5pGFOSYvsCX7guXVl41Rbdo-JLY"
26-Oct-2018 11:24:17.215 general: error: /usr/local/etc/namedb/master/example.com.zone.jnl: create: permission denied
26-Oct-2018 11:24:17.215 update: info: client @0x8031d8a00 127.0.0.1#41221/key letsencrypt: updating zone 'example.com/IN': error: journal open failed: unexpected error

 

CloudFlare API v4 and Fail2ban: Fixing the unban action

This posting is ~5 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

In January 2017, I wrote an article about how to protect your WordPress blog using the WP Fail2Ban plugin, fail2ban on your Linux/ FreeBSD host, and CloudFlare. Back then, the fail2ban was using the CloudFlare API V1, which was already deprecated since November 2016.

Free-Photos/ pixabay.com/ Creative Commons CC0

Although the actions were updated later to use CloudFlare API V4, I still had problems with the unbaning of IP addresses. IP addresses were banned, but the unban action failed. 

This is the unban action, which is included in fail2ban (taken from fail2ban-0.10.3.1 which is shipped with FreeBSD 11.1-RELEASE-p10):

actionunban = curl -s -o /dev/null -X DELETE -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
            https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(curl -s -X GET -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
            'https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1' | cut -d'"' -f6)

And this is the unban action, which finally solved this issue:

actionunban = curl -s -o /dev/null -X DELETE -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
            https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(curl -s -X GET -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
            'https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1' | tr -d '\n' | cut -d'"' -f6)

I found the solution at serverfault.com. The only difference is an additional tr -d ‘\n’  in the last line of the statement. Kudos to Jake for fixing this!

To prevent the action file to being overwritten, you should copy the original cloudflare.conf  located in the  action.d  directory, e.g. to mycloudflare.conf , and use the copied action file in your fail definition.

[wordpress]
enabled  = true
filter   = wordpress
logpath  = /var/log/messages
action   = mycloudflare

Simplemonitor – Python-based monitoring

This posting is ~6 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

While searching for a simple monitoring für my root servers, I’m stumbled over a python-based software called Simplemonitor. Other alternatives, like Nagios, or forks like Incinga etc., were a bit too much for my needs.

What is SimpleMonitor?

SimpleMonitor is a Python script which monitors hosts and network connectivity. It is designed to be quick and easy to set up and lacks complex features that can make things like Nagios, OpenNMS and Zenoss overkill for a small business or home network. Remote monitor instances can send their results back to a central location.

My requirements were simple:

  • Ping monitoring
  • TCP monitoring
  • HTTP monitoring
  • Service monitoring
  • Disk space monitoring

Monitoring is nothing without alerting, so I was pretty happy that Simplemonitor is able to send messages into a Slack channel! But it can also send e-mails, SMS, or it can write into a log file. To get a full feature overview, visit the Simplemonitor website.

The project is hosted on GitHub. If you are familiar with Python, you can contribute to the project, or you can add features as you need.

Installation & configuration

The installation is pretty simple: Just fetch the ZIP or the tarball from the project website, and extract it.

The configuration is split into two files:

  • monitor.ini
  • monitors.ini

The naming is a bit confusing. The monitor.ini contains the basic monitoring configuration, like the interval for the checks, the alerting and reporting settings. The monitors.ini contains the configuration of the service checks. That’s confusing, that confused me, and so I changed the name of the monitors.ini to services.ini.

[monitor]
interval=60
monitors=services.ini

The services.ini (monitors.ini) contains the service checks. This is a short example of a ping, a service check, a port check, and a disk space check.

[ping-host1]
type=host
host=host1.tld.de
tolerance=3

[svc-postfix-host1]
type=rc
runon=host1.tld.de
service=postfix

[port-postfix-host1]
type=tcp
host=host1.tld.de
port=25

[diskspace]
type=diskspace
partition=/
limit=4096M

The alerting is configured in the monitor.ini. I’m using only the Slack notification. All you need is a web hook and the corresponding web hook URL.

[slack]
type=slack
channel=#monitoring
limit=1
url=https://hooks.slack.com/services/afjnsdifnsdfnsdf

In case of a service fail, or service recovery, a notification is sent to the configured Slack channel.

To start Simplemonitor, just start the monitor.py. It expects the monitor.ini in the same directory.

root@host1 /opt/simplemonitor # python2 monitor.py -v
SimpleMonitor v1.7
--> Loading main config from monitor.ini
--> Loading monitor config from services.ini
Adding host monitor ping-host2
Adding rc monitor svc-postfix-host1
Adding rc monitor svc-nginx-host1
Adding rc monitor svc-mysql-host1
Adding rc monitor svc-fail2ban-host1
Adding rc monitor svc-postgrey-host1
Adding rc monitor svc-phpfpm-host1
Adding rc monitor svc-named-host1
Adding diskspace monitor diskspace
--> Loaded 9 monitors.

Adding logfile logger logfile
Adding slack alerter slack

--> Starting... (loop runs every 60s) Hit ^C to stop
php_fpm is running as pid 33937.
Passed: svc-phpfpm-host1
named is running as pid 566.
Passed: svc-named-host1
fail2ban is running as pid 41306.
Passed: svc-fail2ban-host1
Passed: diskspace
postgrey is running as pid 649.
Passed: svc-postgrey-host1
mysql is running as pid 23726.
Passed: svc-mysql-host1
Passed: ping-host2
postfix is running as pid 53332.
Passed: svc-postfix-host1
nginx is running as pid 52736.
Passed: svc-nginx-host1

Summary

I really like the simplicity of Simplemonitor. Download, extract, configure, run, done. That’s what I’ve searched for. It is still under development, but you should not expect that it will gain much complexity. Even if features will be added, it should be a simple monitoring.

Stunnel and Squid on FreeBSD 11

This posting is ~6 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

I don’t like to use untrusted networks. When I have to use such a network, e.g. an open WiFi network, I use a TLS encrypted tunnel connection to encrypt all web traffic that travels through the untrusted network. I’m using a simple stunnel/ Squid setup for this. My setup consists of three components:

  • Stunnel (server mode)
  • Squid proxy
  • Stunnel (client mode)

What is stunnel?

Stunnel is an OSS project that uses OpenSSL to encrypt traffic. The website describes Stunnel as follows:

Stunnel is a proxy designed to add TLS encryption functionality to existing clients and servers without any changes in the programs’ code. Its architecture is optimized for security, portability, and scalability (including load-balancing), making it suitable for large deployments.

How it works

The traffic flow looks like this:

Stunnel Secure Tunnel Connection Diagram

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

The browser connects to the Stunnel client on 127.0.0.1:8080. This is done by configuring 127.0.0.1:8080 as proxy server in the browser. The traffic enters the tunnel on the client-side, and Stunnel opens a connection to the server-side. You can use any port, as long as it is unused on the server-side. I use 443/tcp. The connection is encrypted using TLS, and the connection is authenticated by a pre-shared key (PSK). On the server, the traffic leaves the tunnel, and the connection attempt of the client is directed to the Squid proxy, which listens on 127.0.0.1:3128 for connections. Summarized, my browser connectes the Squid proxy on my FreeBSD host over a TLS encrypted connection.

Installation and configuration on FreeBSD

Stunnel and Squid can be installed using pkg install .

root@server:~ # pkg search squid-3.5
squid-3.5.24_2                 HTTP Caching Proxy
root@server:~ # pkg search stunnel
stunnel-5.41,1                 SSL encryption wrapper for standard network daemons

The configuration files are located under /usr/local/etc/stunnel and /usr/local/etc/squid. After the installation of stunnel, an additional directory for the PID file must be created. Stunnel is not running with root privileges, thus it can’t create its PID file in /var/run.

root@server:/var/run # mkdir /var/run/stunnel/
root@server:/var/run # chown stunnel:stunnel /var/run/stunnel

The stunnel.conf is pretty simple. I’m using a Let’s Encrypt certificate on the server-side. If you like, you can create your own certificate using OpenSSL. But I prefer Let’s Encrypt.

cert = /usr/local/etc/letsencrypt/live/server/fullchain.pem
key = /usr/local/etc/letsencrypt/live/server/privkey.pem
pid = /var/run/stunnel/stunnel.pid
setuid = stunnel
setgid = stunnel
sslVersion = TLSv1.2
debug = 3
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
verify = 2
compression = deflate

[Tunnel.Web]
accept = 46.x.x.x:443
connect = 127.0.0.1:8080
ciphers = PSK
PSKsecrets = /usr/local/etc/stunnel/psk.txt
CAFile = /usr/local/etc/letsencrypt/live/server/fullchain.pem

The psk.txt contains the pre-shared key. The same file must be located on the client-side. The file itself it pretty simple – username:passphrase. Make sure that the PSK file is not group- and world-readable!

patrick:SuperSecretPassw0rd

The squid.conf is also pretty simple. Make sure that Squid only listens on localhost! I disabled the access log. I simply don’t need it, because I’m the only user. And I don’t have to rotate another logfile. Some ACLs of Squid are now implicitly active. There is no need to configure localhsot or 127.0.0.1 as a source, if you want to allow http access only from localhost. Make sure, that all requests are only allowed from localhost!

acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl Safe_ports port 2222
acl Safe_ports port 8443
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access deny all
icp_access deny all
htcp_access deny all
http_port 127.0.0.1:8080
cache_mem 1024 MB
maximum_object_size_in_memory 8 MB
cache_dir ufs /var/squid/cache 1024 16 256 no-store
minimum_object_size 0 KB
maximum_object_size 8192 KB
cache_swap_low 95
cache_swap_high 98
logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
# access_log /var/log/squid/access.log combined
access_log none
cache_log /dev/null
cache_store_log /dev/null
ftp_user joe.doe@gmail.com
htcp_port 0
coredump_dir /var/squid/cache
visible_hostname proxy

To enable stunnel and squid in the /etc/rc.conf, add the following lines to your /etc/rc.conf. The stunnel_pidfile  option tells Stunnel, where it should create its PID file.

squid_enable="YES"
stunnel_enable="YES"
stunnel_pidfile="/var/run/stunnel/stunnel.pid"

Make sure that you have initialized the Squid cache dir, before you start squid. Initialize the cache dir, and start Squid and Stunnel on the server-side.

Installation and configuration on Windows

On the client-side, you have to install Stunnel. You can fine installer files for Windows on stunnel.org. The config of the client is pretty simple. The psk.txt contain the same username and passphrase as on the server-side. The file must be located in the same directory as the stunnel.conf on the client.

socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
client = yes
sslVersion = TLSv1.2
debug=2

[Tunnel.Web]
accept = localhost:8080
connect = 46.x.x.x:443
PSKsecrets = psk.txt

Test your connection

Start Stunnel on your client and configure 127.0.0.1:8080 as proxy in your browser. If you access https://www.whatismyip.com, you should see the IP address of your server, not the IP address of your local internet connection.

You can check the encrypted connection with Wireshark on the client-side, or with tcpdump on the server-side.

Please note, that the connection is only encrypted until it hits your server. Traffic that leaves your server, e.g. HTTP requests, are unencrypted. It is only an encrypted connection to your proxy, not and encrypted end-2-end connection.